2023-12-11T19:01:19.835 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-11T19:01:19.835 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203 --verbose --name rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi --owner scheduled_rishabh@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} -- /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/orig.config.yaml 2023-12-11T19:01:19.892 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-11T19:01:19.986 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203 branch: rishabh-2023dec11 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7487203' kernel: flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-2023dec11 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: a62fda73efba5520d409bb5a21bd8373289dda5d subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: a62fda73efba5520d409bb5a21bd8373289dda5d kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023dec11 sha1: a62fda73efba5520d409bb5a21bd8373289dda5d owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5791 sha1: a62fda73efba5520d409bb5a21bd8373289dda5d sleep_before_teardown: 0 subset: 209/512 suite: fs suite_branch: rishabh-2023dec11 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a62fda73efba5520d409bb5a21bd8373289dda5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-11_15:37:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-11T19:01:19.986 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa; will attempt to use it 2023-12-11T19:01:19.987 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks 2023-12-11T19:01:19.987 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-11T19:01:19.988 INFO:teuthology.task.internal:Checking packages... 2023-12-11T19:01:20.014 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a62fda73efba5520d409bb5a21bd8373289dda5d' 2023-12-11T19:01:20.014 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-11T19:01:20.014 INFO:teuthology.packaging:ref: None 2023-12-11T19:01:20.014 INFO:teuthology.packaging:tag: None 2023-12-11T19:01:20.014 INFO:teuthology.packaging:branch: rishabh-2023dec11 2023-12-11T19:01:20.014 INFO:teuthology.packaging:sha1: a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:01:20.015 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-2023dec11 2023-12-11T19:01:20.233 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-119.ga62fda73 2023-12-11T19:01:20.234 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-11T19:01:20.241 INFO:teuthology.task.internal:no buildpackages task found 2023-12-11T19:01:20.241 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-11T19:01:20.249 INFO:teuthology.task.internal:Saving configuration 2023-12-11T19:01:20.269 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-11T19:01:20.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T19:01:20.301 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 18:54:46.448960', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T19:01:20.324 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 18:54:46.450660', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T19:01:20.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 18:54:46.452004', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T19:01:20.346 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-11T19:01:20.352 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-11T19:01:20.352 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-11T19:01:20.352 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-11T19:01:20.352 INFO:teuthology.run_tasks:Running task console_log... 2023-12-11T19:01:20.454 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6807682dc0>, signals=[15]) 2023-12-11T19:01:20.454 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-11T19:01:20.460 INFO:teuthology.task.internal:Opening connections... 2023-12-11T19:01:20.461 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:01:20.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:01:20.530 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2023-12-11T19:01:20.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:01:20.598 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2023-12-11T19:01:20.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:01:20.665 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-11T19:01:20.672 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2023-12-11T19:01:20.693 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2023-12-11T19:01:20.693 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2023-12-11T19:01:20.753 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Red Hat Enterprise Linux" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8.6 (Ootpa)" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:ID="rhel" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="fedora" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8.6" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.redhat.com/" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-11T19:01:20.754 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:01:20.755 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-11T19:01:20.755 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-11T19:01:20.755 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-11T19:01:20.755 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-11T19:01:20.755 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2023-12-11T19:01:20.781 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2023-12-11T19:01:20.802 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2023-12-11T19:01:20.802 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2023-12-11T19:01:20.861 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Red Hat Enterprise Linux" 2023-12-11T19:01:20.861 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8.6 (Ootpa)" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:ID="rhel" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="fedora" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8.6" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.redhat.com/" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-11T19:01:20.862 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-11T19:01:20.863 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-11T19:01:20.863 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-12-11T19:01:20.889 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-12-11T19:01:20.910 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-12-11T19:01:20.910 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-12-11T19:01:20.968 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2023-12-11T19:01:20.969 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-11T19:01:20.970 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-11T19:01:20.970 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:01:20.970 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-11T19:01:20.970 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-11T19:01:20.970 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-11T19:01:20.970 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-11T19:01:20.970 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-12-11T19:01:20.989 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-11T19:01:20.999 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-11T19:01:21.007 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-11T19:01:21.007 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2023-12-11T19:01:21.009 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2023-12-11T19:01:21.011 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2023-12-11T19:01:21.030 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-11T19:01:21.036 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-11T19:01:21.036 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2023-12-11T19:01:21.070 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2023-12-11T19:01:21.072 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2023-12-11T19:01:21.105 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-11T19:01:21.182 INFO:teuthology.run_tasks:Running task kernel... 2023-12-11T19:01:21.197 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-11T19:01:21.198 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}}, timeout 300 2023-12-11T19:01:21.198 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-11T19:01:21.199 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-11T19:01:21.199 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-11T19:01:21.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:01:21.218 DEBUG:teuthology.task.kernel:sha1 for mon.c is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.218 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-11T19:01:21.218 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-12-11T19:01:21.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:01:21.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:01:21.220 DEBUG:teuthology.task.kernel:sha1 for mon.a is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.220 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-11T19:01:21.220 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2023-12-11T19:01:21.220 DEBUG:teuthology.task.kernel:sha1 for mon.b is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.221 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-11T19:01:21.221 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-12-11T19:01:21.276 INFO:teuthology.orchestra.run.smithi167.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-11T19:01:21.276 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.277 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-11T19:01:21.277 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.281 INFO:teuthology.orchestra.run.smithi066.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-11T19:01:21.281 INFO:teuthology.orchestra.run.smithi070.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-11T19:01:21.281 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.281 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-11T19:01:21.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.283 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.283 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-11T19:01:21.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:21.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum upgrade -y linux-firmware 2023-12-11T19:01:21.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum upgrade -y linux-firmware 2023-12-11T19:01:21.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum upgrade -y linux-firmware 2023-12-11T19:01:22.569 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-12-11T19:01:22.570 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2023-12-11T19:01:22.599 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-12-11T19:01:22.599 INFO:teuthology.orchestra.run.smithi070.stdout:Unable to read consumer identity 2023-12-11T19:01:22.675 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-12-11T19:01:22.676 INFO:teuthology.orchestra.run.smithi066.stdout:Unable to read consumer identity 2023-12-11T19:01:24.018 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-12-11T19:01:24.080 INFO:teuthology.orchestra.run.smithi066.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-12-11T19:01:24.142 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-12-11T19:01:25.493 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-12-11T19:01:25.882 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 16 MB 00:01 2023-12-11T19:01:25.982 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 16 MB 00:01 2023-12-11T19:01:29.010 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 445 kB/s | 24 kB 00:00 2023-12-11T19:01:29.078 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:01 ago on Mon 11 Dec 2023 07:01:28 PM UTC. 2023-12-11T19:01:29.402 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 431 kB/s | 24 kB 00:00 2023-12-11T19:01:29.724 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-12-11T19:01:29.725 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2023-12-11T19:01:29.726 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-12-11T19:01:29.782 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 593 kB/s | 24 kB 00:00 2023-12-11T19:01:29.867 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.b... 2023-12-11T19:01:29.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:30.017 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ 2023-12-11T19:01:30.017 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-11T19:01:30.019 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi070.front.sepia.ceph.com to complete... 2023-12-11T19:01:30.162 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-12-11T19:01:30.163 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-12-11T19:01:30.163 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-12-11T19:01:30.286 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.c... 2023-12-11T19:01:30.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:30.432 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ 2023-12-11T19:01:30.433 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-11T19:01:30.435 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi167.front.sepia.ceph.com to complete... 2023-12-11T19:01:30.562 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-12-11T19:01:30.563 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2023-12-11T19:01:30.563 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-12-11T19:01:30.712 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.a... 2023-12-11T19:01:30.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-11T19:01:30.855 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ 2023-12-11T19:01:30.856 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-11T19:01:30.858 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi066.front.sepia.ceph.com to complete... 2023-12-11T19:01:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11 19:01:48 URL:https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [383180176/383180176] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-11T19:01:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:FINISHED --2023-12-11 19:01:48-- 2023-12-11T19:01:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:Total wall clock time: 18s 2023-12-11T19:01:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:Downloaded: 1 files, 365M in 18s (20.0 MB/s) 2023-12-11T19:01:48.525 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.b... 2023-12-11T19:01:48.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-11T19:01:50.250 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying... ######################################## 2023-12-11T19:01:50.331 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing... ######################################## 2023-12-11T19:01:50.418 INFO:teuthology.orchestra.run.smithi070.stdout:Updating / installing... 2023-12-11T19:02:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11 19:02:05 URL:https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [383180176/383180176] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-11T19:02:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:FINISHED --2023-12-11 19:02:05-- 2023-12-11T19:02:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:Total wall clock time: 35s 2023-12-11T19:02:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:Downloaded: 1 files, 365M in 34s (10.6 MB/s) 2023-12-11T19:02:05.138 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.c... 2023-12-11T19:02:05.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-11T19:02:06.725 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying... ######################################## 2023-12-11T19:02:06.795 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing... ######################################## 2023-12-11T19:02:06.885 INFO:teuthology.orchestra.run.smithi167.stdout:Updating / installing... 2023-12-11T19:02:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11 19:02:08 URL:https://1.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/8/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [383180176/383180176] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-11T19:02:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:FINISHED --2023-12-11 19:02:08-- 2023-12-11T19:02:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:Total wall clock time: 37s 2023-12-11T19:02:08.119 INFO:teuthology.orchestra.run.smithi066.stderr:Downloaded: 1 files, 365M in 37s (9.93 MB/s) 2023-12-11T19:02:08.122 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.a... 2023-12-11T19:02:08.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-11T19:02:09.799 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying... ######################################## 2023-12-11T19:02:09.860 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing... ######################################## 2023-12-11T19:02:09.956 INFO:teuthology.orchestra.run.smithi066.stdout:Updating / installing... 2023-12-11T19:02:28.960 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-11T19:02:30.480 INFO:teuthology.orchestra.run.smithi070.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-11T19:02:45.071 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-11T19:02:46.607 INFO:teuthology.orchestra.run.smithi167.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-11T19:02:50.730 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-11T19:02:54.531 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-11T19:02:54.531 DEBUG:teuthology.orchestra.run.smithi070:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-11T19:02:54.599 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.599 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.599 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-11T19:02:54.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-11T19:02:54.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-11T19:02:54.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-11T19:02:54.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-11T19:02:54.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-11T19:02:54.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-11T19:02:54.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-11T19:02:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-11T19:02:54.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-11T19:02:54.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-11T19:02:54.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-11T19:02:54.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-11T19:02:54.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-11T19:02:54.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-11T19:02:54.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-11T19:02:54.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-11T19:02:54.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-11T19:02:54.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-11T19:02:54.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-11T19:02:54.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-11T19:02:54.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-11T19:02:54.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-11T19:02:54.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-11T19:02:54.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-11T19:02:54.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-11T19:02:54.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-11T19:02:54.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-11T19:02:54.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-11T19:02:54.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-11T19:02:54.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-11T19:02:54.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-11T19:02:54.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-11T19:02:54.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-11T19:02:54.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-11T19:02:54.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-11T19:02:54.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-11T19:02:54.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-11T19:02:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-11T19:02:54.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-11T19:02:54.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-11T19:02:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-11T19:02:54.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-11T19:02:54.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-11T19:02:54.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-11T19:02:54.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-11T19:02:54.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-11T19:02:54.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-11T19:02:54.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-11T19:02:54.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-11T19:02:54.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-11T19:02:54.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-11T19:02:54.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-11T19:02:54.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-11T19:02:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-11T19:02:54.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-11T19:02:54.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-11T19:02:54.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-11T19:02:54.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-11T19:02:54.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-11T19:02:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-11T19:02:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-11T19:02:54.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-11T19:02:54.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-11T19:02:54.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-11T19:02:54.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-11T19:02:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-11T19:02:54.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-11T19:02:54.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-11T19:02:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-11T19:02:54.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-11T19:02:54.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-11T19:02:54.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-11T19:02:54.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-11T19:02:54.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-11T19:02:54.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-11T19:02:54.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-11T19:02:54.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-11T19:02:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-11T19:02:54.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-11T19:02:54.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-11T19:02:54.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-11T19:02:54.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-11T19:02:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-11T19:02:54.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-11T19:02:54.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-11T19:02:54.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-11T19:02:54.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-11T19:02:54.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-11T19:02:54.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-11T19:02:54.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-11T19:02:54.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-11T19:02:54.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-11T19:02:54.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-11T19:02:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-11T19:02:54.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-11T19:02:54.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-11T19:02:54.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-11T19:02:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-11T19:02:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-11T19:02:54.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-11T19:02:54.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-11T19:02:54.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-11T19:02:54.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-11T19:02:54.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-11T19:02:54.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-11T19:02:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-11T19:02:54.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-11T19:02:54.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-11T19:02:54.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-11T19:02:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-11T19:02:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-11T19:02:54.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-11T19:02:54.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-11T19:02:54.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-11T19:02:54.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-11T19:02:54.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-11T19:02:54.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-11T19:02:54.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-11T19:02:54.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-11T19:02:54.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-11T19:02:54.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-11T19:02:54.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-11T19:02:54.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-11T19:02:54.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-11T19:02:54.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-11T19:02:54.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-11T19:02:54.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-11T19:02:54.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-11T19:02:54.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-11T19:02:54.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-11T19:02:54.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-11T19:02:54.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-11T19:02:54.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-11T19:02:54.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-11T19:02:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-11T19:02:54.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-11T19:02:54.730 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.730 DEBUG:teuthology.orchestra.run.smithi070:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:postinstall scriptlet (using /bin/sh): 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-11T19:02:54.757 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.758 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-12-11T19:02:54.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2023-12-11T19:02:54.837 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.02 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 123.el8 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Group : System Environment/Base 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 9265450 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-11T19:02:54.838 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Relocations : (not relocatable) 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : Red Hat, Inc. 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : Red Hat, Inc. 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:02:54.839 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2023-12-11T19:02:54.840 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-11T19:02:54.840 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 6.6.0-gcd68252d7b4d 2023-12-11T19:02:54.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-11T19:02:55.029 INFO:teuthology.orchestra.run.smithi066.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-11T19:02:57.762 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2023-12-11T19:03:00.021 INFO:teuthology.orchestra.run.smithi070.stderr:done 2023-12-11T19:03:00.023 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2023-12-11T19:03:00.039 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.yanewECrIF 2023-12-11T19:03:00.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yanewECrIF 2023-12-11T19:03:00.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.yanewECrIF 2023-12-11T19:03:00.270 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.yanewECrIF is 6KB 2023-12-11T19:03:00.318 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.yanewECrIF 2023-12-11T19:03:00.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2023-12-11T19:03:00.399 INFO:teuthology.orchestra.run.smithi070.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-12-11T19:03:00.399 INFO:teuthology.orchestra.run.smithi070.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-12-11T19:03:00.399 INFO:teuthology.orchestra.run.smithi070.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-gcd68252d7b4d.conf 2023-12-11T19:03:00.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-gcd68252d7b4d 2023-12-11T19:03:00.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2023-12-11T19:03:10.053 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-11T19:03:10.054 DEBUG:teuthology.orchestra.run.smithi167:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-11T19:03:10.109 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.109 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.109 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-11T19:03:10.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-11T19:03:10.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-11T19:03:10.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-11T19:03:10.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-11T19:03:10.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-11T19:03:10.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-11T19:03:10.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-11T19:03:10.117 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-11T19:03:10.118 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-11T19:03:10.119 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-11T19:03:10.120 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-11T19:03:10.121 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-11T19:03:10.122 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-11T19:03:10.123 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-11T19:03:10.124 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-11T19:03:10.125 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-11T19:03:10.126 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-11T19:03:10.127 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-11T19:03:10.128 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-11T19:03:10.129 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-11T19:03:10.130 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-11T19:03:10.131 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-11T19:03:10.132 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-11T19:03:10.133 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-11T19:03:10.134 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-11T19:03:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-11T19:03:10.136 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-11T19:03:10.137 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-11T19:03:10.138 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-11T19:03:10.139 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-11T19:03:10.140 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-11T19:03:10.141 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-11T19:03:10.142 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-11T19:03:10.143 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-11T19:03:10.144 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-11T19:03:10.145 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-11T19:03:10.146 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-11T19:03:10.147 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-11T19:03:10.148 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-11T19:03:10.149 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-11T19:03:10.150 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-11T19:03:10.151 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-11T19:03:10.153 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-11T19:03:10.153 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-11T19:03:10.154 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-11T19:03:10.155 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-11T19:03:10.156 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-11T19:03:10.157 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-11T19:03:10.158 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-11T19:03:10.159 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-11T19:03:10.160 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-11T19:03:10.161 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-11T19:03:10.162 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-11T19:03:10.163 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-11T19:03:10.164 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-11T19:03:10.165 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-11T19:03:10.166 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-11T19:03:10.167 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-11T19:03:10.168 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-11T19:03:10.169 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-11T19:03:10.170 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-11T19:03:10.171 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-11T19:03:10.172 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-11T19:03:10.173 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-11T19:03:10.174 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-11T19:03:10.175 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-11T19:03:10.176 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-11T19:03:10.177 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-11T19:03:10.178 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-11T19:03:10.179 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-11T19:03:10.180 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-11T19:03:10.181 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-11T19:03:10.182 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-11T19:03:10.183 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-11T19:03:10.184 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-11T19:03:10.185 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-11T19:03:10.186 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-11T19:03:10.187 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-11T19:03:10.188 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-11T19:03:10.189 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-11T19:03:10.190 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-11T19:03:10.191 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-11T19:03:10.192 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-11T19:03:10.193 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-11T19:03:10.194 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-11T19:03:10.195 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-11T19:03:10.196 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-11T19:03:10.197 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-11T19:03:10.198 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-11T19:03:10.199 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-11T19:03:10.200 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-11T19:03:10.201 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-11T19:03:10.202 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-11T19:03:10.203 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-11T19:03:10.204 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-11T19:03:10.205 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-11T19:03:10.206 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-11T19:03:10.207 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-11T19:03:10.208 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-11T19:03:10.209 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-11T19:03:10.210 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-11T19:03:10.211 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-11T19:03:10.212 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-11T19:03:10.213 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-11T19:03:10.214 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-11T19:03:10.215 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-11T19:03:10.216 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-11T19:03:10.217 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-11T19:03:10.218 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-11T19:03:10.219 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-11T19:03:10.220 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-11T19:03:10.221 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-11T19:03:10.222 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-11T19:03:10.223 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-11T19:03:10.224 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-11T19:03:10.225 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-11T19:03:10.226 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-11T19:03:10.227 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-11T19:03:10.228 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-11T19:03:10.229 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-11T19:03:10.230 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-11T19:03:10.231 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-11T19:03:10.232 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-11T19:03:10.233 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-11T19:03:10.234 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-11T19:03:10.235 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-11T19:03:10.236 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-11T19:03:10.237 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-11T19:03:10.238 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-11T19:03:10.239 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-11T19:03:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-11T19:03:10.241 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-11T19:03:10.242 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-11T19:03:10.243 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-11T19:03:10.244 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-11T19:03:10.245 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-11T19:03:10.246 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.246 DEBUG:teuthology.orchestra.run.smithi167:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-11T19:03:10.271 INFO:teuthology.orchestra.run.smithi167.stdout:postinstall scriptlet (using /bin/sh): 2023-12-11T19:03:10.271 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-11T19:03:10.271 INFO:teuthology.orchestra.run.smithi167.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:10.271 INFO:teuthology.orchestra.run.smithi167.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.272 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2023-12-11T19:03:10.273 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.02 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 123.el8 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-12-11T19:03:10.351 INFO:teuthology.orchestra.run.smithi167.stdout:Group : System Environment/Base 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 9265450 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Relocations : (not relocatable) 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : Red Hat, Inc. 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : Red Hat, Inc. 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-11T19:03:10.352 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-11T19:03:10.353 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-11T19:03:10.353 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2023-12-11T19:03:10.353 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:03:10.353 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2023-12-11T19:03:10.354 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-11T19:03:10.354 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 6.6.0-gcd68252d7b4d 2023-12-11T19:03:10.354 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-11T19:03:13.893 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2023-12-11T19:03:15.834 INFO:teuthology.orchestra.run.smithi167.stderr:done 2023-12-11T19:03:15.835 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-12-11T19:03:15.861 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.BT9qv4bGWZ 2023-12-11T19:03:15.861 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BT9qv4bGWZ 2023-12-11T19:03:15.939 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.BT9qv4bGWZ 2023-12-11T19:03:16.100 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.BT9qv4bGWZ is 6KB 2023-12-11T19:03:16.148 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.BT9qv4bGWZ 2023-12-11T19:03:16.164 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2023-12-11T19:03:16.229 INFO:teuthology.orchestra.run.smithi167.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-12-11T19:03:16.229 INFO:teuthology.orchestra.run.smithi167.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-12-11T19:03:16.229 INFO:teuthology.orchestra.run.smithi167.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-gcd68252d7b4d.conf 2023-12-11T19:03:16.231 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-gcd68252d7b4d 2023-12-11T19:03:16.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2023-12-11T19:03:18.486 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-11T19:03:18.486 DEBUG:teuthology.orchestra.run.smithi066:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-11T19:03:18.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-11T19:03:18.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-11T19:03:18.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-11T19:03:18.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-11T19:03:18.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-11T19:03:18.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-11T19:03:18.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-11T19:03:18.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-11T19:03:18.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-11T19:03:18.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-11T19:03:18.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-11T19:03:18.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-11T19:03:18.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-11T19:03:18.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-11T19:03:18.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-11T19:03:18.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-11T19:03:18.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-11T19:03:18.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-11T19:03:18.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-11T19:03:18.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-11T19:03:18.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-11T19:03:18.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-11T19:03:18.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-11T19:03:18.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-11T19:03:18.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-11T19:03:18.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-11T19:03:18.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-11T19:03:18.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-11T19:03:18.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-11T19:03:18.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-11T19:03:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-11T19:03:18.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-11T19:03:18.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-11T19:03:18.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-11T19:03:18.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-11T19:03:18.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-11T19:03:18.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-11T19:03:18.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-11T19:03:18.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-11T19:03:18.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-11T19:03:18.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-11T19:03:18.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-11T19:03:18.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-11T19:03:18.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-11T19:03:18.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-11T19:03:18.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-11T19:03:18.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-11T19:03:18.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-11T19:03:18.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-11T19:03:18.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-11T19:03:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-11T19:03:18.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-11T19:03:18.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-11T19:03:18.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-11T19:03:18.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-11T19:03:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-11T19:03:18.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-11T19:03:18.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-11T19:03:18.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-11T19:03:18.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-11T19:03:18.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-11T19:03:18.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-11T19:03:18.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-11T19:03:18.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-11T19:03:18.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-11T19:03:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-11T19:03:18.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-11T19:03:18.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-11T19:03:18.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-11T19:03:18.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-11T19:03:18.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-11T19:03:18.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-11T19:03:18.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-11T19:03:18.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-11T19:03:18.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-11T19:03:18.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-11T19:03:18.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-11T19:03:18.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-11T19:03:18.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-11T19:03:18.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-11T19:03:18.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-11T19:03:18.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-11T19:03:18.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-11T19:03:18.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-11T19:03:18.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-11T19:03:18.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-11T19:03:18.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-11T19:03:18.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-11T19:03:18.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-11T19:03:18.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-11T19:03:18.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-11T19:03:18.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-11T19:03:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-11T19:03:18.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-11T19:03:18.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-11T19:03:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-11T19:03:18.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-11T19:03:18.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-11T19:03:18.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-11T19:03:18.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-11T19:03:18.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-11T19:03:18.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-11T19:03:18.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-11T19:03:18.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-11T19:03:18.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-11T19:03:18.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-11T19:03:18.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-11T19:03:18.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-11T19:03:18.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-11T19:03:18.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-11T19:03:18.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-11T19:03:18.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-11T19:03:18.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-11T19:03:18.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-11T19:03:18.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-11T19:03:18.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-11T19:03:18.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-11T19:03:18.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-11T19:03:18.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-11T19:03:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-11T19:03:18.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-11T19:03:18.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-11T19:03:18.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-11T19:03:18.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-11T19:03:18.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-11T19:03:18.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-11T19:03:18.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-11T19:03:18.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-11T19:03:18.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-11T19:03:18.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-11T19:03:18.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-11T19:03:18.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-11T19:03:18.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-11T19:03:18.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-11T19:03:18.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-11T19:03:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-11T19:03:18.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-11T19:03:18.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-11T19:03:18.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-11T19:03:18.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-11T19:03:18.682 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.682 DEBUG:teuthology.orchestra.run.smithi066:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-11T19:03:18.711 INFO:teuthology.orchestra.run.smithi066.stdout:postinstall scriptlet (using /bin/sh): 2023-12-11T19:03:18.711 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-11T19:03:18.711 INFO:teuthology.orchestra.run.smithi066.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:18.711 INFO:teuthology.orchestra.run.smithi066.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:18.711 INFO:teuthology.orchestra.run.smithi066.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.711 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.712 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2023-12-11T19:03:18.713 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -qi grub2-tools 2023-12-11T19:03:18.899 INFO:teuthology.orchestra.run.smithi066.stdout:Name : grub2-tools 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Epoch : 1 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Version : 2.02 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Release : 123.el8 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: x86_64 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Group : System Environment/Base 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Size : 9265450 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:License : GPLv3+ 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-11T19:03:18.900 INFO:teuthology.orchestra.run.smithi066.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout:Relocations : (not relocatable) 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout:Packager : Red Hat, Inc. 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout:Vendor : Red Hat, Inc. 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout:Summary : Support tools for GRUB. 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout:Description : 2023-12-11T19:03:18.901 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:03:18.902 INFO:teuthology.orchestra.run.smithi066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-11T19:03:18.902 INFO:teuthology.orchestra.run.smithi066.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-11T19:03:18.902 INFO:teuthology.orchestra.run.smithi066.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-11T19:03:18.902 INFO:teuthology.orchestra.run.smithi066.stdout:hardware devices. 2023-12-11T19:03:18.902 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:03:18.902 INFO:teuthology.orchestra.run.smithi066.stdout:This subpackage provides tools for support of all platforms. 2023-12-11T19:03:18.903 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-11T19:03:18.903 INFO:teuthology.task.kernel:Updating grub on smithi066 to boot 6.6.0-gcd68252d7b4d 2023-12-11T19:03:18.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-11T19:03:21.309 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2023-12-11T19:03:23.740 INFO:teuthology.orchestra.run.smithi066.stderr:done 2023-12-11T19:03:23.741 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2023-12-11T19:03:23.758 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.Pjhy1sH7Ia 2023-12-11T19:03:23.758 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Pjhy1sH7Ia 2023-12-11T19:03:23.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.Pjhy1sH7Ia 2023-12-11T19:03:23.988 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.Pjhy1sH7Ia is 6KB 2023-12-11T19:03:24.035 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.Pjhy1sH7Ia 2023-12-11T19:03:24.051 DEBUG:teuthology.orchestra.run.smithi066:> sudo /bin/ls /boot/loader/entries || true 2023-12-11T19:03:24.179 INFO:teuthology.orchestra.run.smithi066.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-12-11T19:03:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-12-11T19:03:24.180 INFO:teuthology.orchestra.run.smithi066.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-gcd68252d7b4d.conf 2023-12-11T19:03:24.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-gcd68252d7b4d 2023-12-11T19:03:24.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2023-12-11T19:03:30.570 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-11T19:03:30.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:03:30.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:03:46.399 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2023-12-11T19:03:46.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:03:46.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:03:49.090 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-12-11T19:03:54.306 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-11T19:03:54.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:03:54.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:03:58.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:03:58.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:01.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-12-11T19:04:04.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-11T19:04:12.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-12-11T19:04:13.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:04:13.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:13.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:04:13.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:17.026 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-11T19:04:21.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:04:21.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:24.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-12-11T19:04:29.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:04:29.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:31.586 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-12-11T19:04:36.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:04:36.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:39.778 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-12-11T19:04:46.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:04:46.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:47.174 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-12-11T19:04:47.623 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:04:47.623 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-gcd68252d7b4d"... 2023-12-11T19:04:47.624 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-12-11T19:04:47.665 INFO:teuthology.orchestra.run.smithi070.stdout:6.6.0-gcd68252d7b4d 2023-12-11T19:04:47.665 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-11T19:04:47.665 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-11T19:04:47.665 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-11T19:04:48.666 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-11T19:04:48.667 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-11T19:04:48.814 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2023-12-11T19:04:48.835 DEBUG:teuthology.parallel:result is None 2023-12-11T19:04:54.779 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:04:54.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:04:57.858 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-12-11T19:05:15.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:05:15.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:05:18.914 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-12-11T19:05:29.034 DEBUG:teuthology.orchestra.remote:timed out 2023-12-11T19:05:39.915 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:05:39.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:05:42.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-12-11T19:05:44.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:05:44.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:05:44.402 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-12-11T19:05:44.828 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:05:44.828 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.6.0-gcd68252d7b4d"... 2023-12-11T19:05:44.828 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-12-11T19:05:44.872 INFO:teuthology.orchestra.run.smithi167.stdout:6.6.0-gcd68252d7b4d 2023-12-11T19:05:44.872 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-11T19:05:44.872 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-11T19:05:44.872 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-11T19:05:45.873 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-12-11T19:05:45.873 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-11T19:05:46.012 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2023-12-11T19:05:46.047 DEBUG:teuthology.parallel:result is None 2023-12-11T19:06:06.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:06:06.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:06:07.333 DEBUG:teuthology.orchestra.run.smithi066:> true 2023-12-11T19:06:07.722 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:06:07.722 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-gcd68252d7b4d"... 2023-12-11T19:06:07.722 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2023-12-11T19:06:07.767 INFO:teuthology.orchestra.run.smithi066.stdout:6.6.0-gcd68252d7b4d 2023-12-11T19:06:07.767 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-11T19:06:07.767 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-11T19:06:07.767 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-11T19:06:08.768 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-11T19:06:08.768 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-11T19:06:08.913 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2023-12-11T19:06:08.936 DEBUG:teuthology.parallel:result is None 2023-12-11T19:06:08.937 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-11T19:06:08.944 INFO:teuthology.task.internal:Creating test directory... 2023-12-11T19:06:08.945 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-11T19:06:08.949 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-11T19:06:08.956 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-11T19:06:09.006 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-11T19:06:09.013 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-11T19:06:09.023 INFO:teuthology.task.internal:Creating archive directory... 2023-12-11T19:06:09.023 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-11T19:06:09.034 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-11T19:06:09.039 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-11T19:06:09.099 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-11T19:06:09.106 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-11T19:06:09.107 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-11T19:06:09.111 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-11T19:06:09.115 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-11T19:06:09.188 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T19:06:09.190 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T19:06:09.190 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T19:06:09.217 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T19:06:09.218 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T19:06:09.219 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T19:06:09.223 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-11T19:06:09.231 INFO:teuthology.task.internal:Configuring sudo... 2023-12-11T19:06:09.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-11T19:06:09.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-11T19:06:09.239 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-11T19:06:09.308 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-11T19:06:09.317 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-11T19:06:09.317 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-11T19:06:09.322 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-11T19:06:09.326 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-11T19:06:09.369 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T19:06:09.476 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T19:06:09.547 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T19:06:09.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T19:06:09.689 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:06:09.689 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-11T19:06:09.759 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T19:06:09.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T19:06:09.908 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T19:06:09.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T19:06:10.046 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:06:10.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-11T19:06:10.114 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T19:06:10.187 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T19:06:10.269 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T19:06:10.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T19:06:10.407 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:06:10.407 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-11T19:06:10.476 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2023-12-11T19:06:10.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2023-12-11T19:06:10.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2023-12-11T19:06:10.597 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T19:06:10.599 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T19:06:10.636 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T19:06:11.055 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-11T19:06:11.064 INFO:teuthology.task.internal:Starting timer... 2023-12-11T19:06:11.064 INFO:teuthology.run_tasks:Running task pcp... 2023-12-11T19:06:11.074 INFO:teuthology.run_tasks:Running task selinux... 2023-12-11T19:06:11.082 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-12-11T19:06:11.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2023-12-11T19:06:11.202 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2023-12-11T19:06:11.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2023-12-11T19:06:11.336 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2023-12-11T19:06:11.341 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2023-12-11T19:06:11.465 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2023-12-11T19:06:11.471 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-11T19:06:11.471 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2023-12-11T19:06:11.520 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2023-12-11T19:06:11.520 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2023-12-11T19:06:11.577 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2023-12-11T19:06:11.578 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2023-12-11T19:06:11.632 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2023-12-11T19:06:11.632 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2023-12-11T19:06:11.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T19:06:11.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:06:11.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T19:06:11.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:06:11.780 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T19:06:11.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:06:11.853 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-11T19:06:11.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2023-12-11T19:06:11.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2023-12-11T19:06:11.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2023-12-11T19:06:12.070 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-11T19:06:12.079 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-11T19:06:12.085 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-11T19:06:12.213 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-11T19:06:12.228 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-11T19:06:12.229 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi070.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-11T19:19:08.672 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2023-12-11T19:19:08.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:19:08.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:19:08.765 DEBUG:teuthology.orchestra.run.smithi066:> true 2023-12-11T19:19:08.877 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-12-11T19:19:08.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:19:08.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:19:08.968 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-12-11T19:19:09.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-12-11T19:19:09.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:19:09.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T19:19:09.166 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-12-11T19:19:09.283 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-11T19:19:09.284 INFO:teuthology.run_tasks:Running task clock... 2023-12-11T19:19:09.295 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-11T19:19:09.295 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-11T19:19:09.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T19:19:09.300 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-11T19:19:09.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T19:19:09.304 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-11T19:19:09.304 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T19:19:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-11T19:19:09.375 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-11T19:19:09.401 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-11T19:19:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-11T19:19:09.412 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-11T19:19:09.436 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-11T19:19:09.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2023-12-11T19:19:09.513 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2023-12-11T19:19:09.526 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2023-12-11T19:19:09.529 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2023-12-11T19:19:09.544 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2023-12-11T19:19:09.555 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2023-12-11T19:19:09.563 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-11T19:19:09.580 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-11T19:19:09.589 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-11T19:19:09.596 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-11T19:19:09.616 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-11T19:19:09.623 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-11T19:19:09.688 INFO:teuthology.orchestra.run.smithi070.stderr:bash: ntpq: command not found 2023-12-11T19:19:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T19:19:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-12-11T19:19:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.710 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-12-11T19:19:09.718 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T19:19:09.718 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-12-11T19:19:09.718 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.718 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.718 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.718 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.730 INFO:teuthology.orchestra.run.smithi066.stderr:bash: ntpq: command not found 2023-12-11T19:19:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T19:19:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2023-12-11T19:19:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.738 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:19:09.740 INFO:teuthology.run_tasks:Running task pexec... 2023-12-11T19:19:09.749 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-11T19:19:09.750 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:19:09.750 DEBUG:teuthology.orchestra.run.smithi066:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-11T19:19:09.750 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi070.front.sepia.ceph.com 2023-12-11T19:19:09.750 DEBUG:teuthology.orchestra.run.smithi070:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-11T19:19:09.751 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi167.front.sepia.ceph.com 2023-12-11T19:19:09.751 DEBUG:teuthology.orchestra.run.smithi167:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-11T19:19:10.229 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:10.240 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:10.279 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:11.018 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 144 kB/s | 4.1 kB 00:00 2023-12-11T19:19:11.022 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 148 kB/s | 4.1 kB 00:00 2023-12-11T19:19:11.047 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 149 kB/s | 4.5 kB 00:00 2023-12-11T19:19:11.269 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 141 kB/s | 4.5 kB 00:00 2023-12-11T19:19:11.273 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 136 kB/s | 4.5 kB 00:00 2023-12-11T19:19:11.413 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 135 kB/s | 4.1 kB 00:00 2023-12-11T19:19:13.287 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout:Resetting modules: 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-12-11T19:19:13.288 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:13.387 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-12-11T19:19:13.388 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-12-11T19:19:13.388 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout:Resetting modules: 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-12-11T19:19:13.389 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:13.685 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout:Resetting modules: 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:13.686 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2023-12-11T19:19:13.687 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2023-12-11T19:19:13.687 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:17.539 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-12-11T19:19:17.821 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-12-11T19:19:17.974 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-12-11T19:19:18.126 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:18.418 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:18.597 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:18.851 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 151 kB/s | 4.1 kB 00:00 2023-12-11T19:19:19.101 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 171 kB/s | 4.5 kB 00:00 2023-12-11T19:19:19.156 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 155 kB/s | 4.1 kB 00:00 2023-12-11T19:19:19.339 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 153 kB/s | 4.5 kB 00:00 2023-12-11T19:19:19.390 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 172 kB/s | 4.5 kB 00:00 2023-12-11T19:19:19.717 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 101 kB/s | 4.1 kB 00:00 2023-12-11T19:19:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-12-11T19:19:21.209 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================================ 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================================ 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout:Installing group/module packages: 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2023-12-11T19:19:21.210 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout:Installing module profiles: 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools/common 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling module streams: 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools rhel8 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================================ 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout:Install 10 Packages 2023-12-11T19:19:21.211 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:21.212 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 18 M 2023-12-11T19:19:21.212 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 65 M 2023-12-11T19:19:21.212 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-12-11T19:19:21.308 INFO:teuthology.orchestra.run.smithi070.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 262 kB/s | 25 kB 00:00 2023-12-11T19:19:21.332 INFO:teuthology.orchestra.run.smithi070.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 340 kB/s | 41 kB 00:00 2023-12-11T19:19:21.358 INFO:teuthology.orchestra.run.smithi070.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 645 kB/s | 94 kB 00:00 2023-12-11T19:19:21.383 INFO:teuthology.orchestra.run.smithi070.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-12-11T19:19:21.409 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-12-11T19:19:21.417 INFO:teuthology.orchestra.run.smithi070.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-11T19:19:21.471 INFO:teuthology.orchestra.run.smithi070.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.7 MB/s | 149 kB 00:00 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================================ 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================================ 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout:Installing group/module packages: 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-11T19:19:21.509 INFO:teuthology.orchestra.run.smithi167.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:Installing module profiles: 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools/common 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools rhel8 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================================ 2023-12-11T19:19:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:Install 10 Packages 2023-12-11T19:19:21.511 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:21.511 INFO:teuthology.orchestra.run.smithi070.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.3 MB/s | 213 kB 00:00 2023-12-11T19:19:21.511 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 18 M 2023-12-11T19:19:21.511 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 65 M 2023-12-11T19:19:21.512 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-12-11T19:19:21.580 INFO:teuthology.orchestra.run.smithi167.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-12-11T19:19:21.605 INFO:teuthology.orchestra.run.smithi167.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 269 kB/s | 25 kB 00:00 2023-12-11T19:19:21.630 INFO:teuthology.orchestra.run.smithi167.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 346 kB/s | 41 kB 00:00 2023-12-11T19:19:21.634 INFO:teuthology.orchestra.run.smithi070.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-12-11T19:19:21.656 INFO:teuthology.orchestra.run.smithi167.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 652 kB/s | 49 kB 00:00 2023-12-11T19:19:21.689 INFO:teuthology.orchestra.run.smithi167.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-11T19:19:21.723 INFO:teuthology.orchestra.run.smithi167.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-12-11T19:19:21.768 INFO:teuthology.orchestra.run.smithi070.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 24 MB/s | 7.0 MB 00:00 2023-12-11T19:19:21.782 INFO:teuthology.orchestra.run.smithi167.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.3 MB/s | 213 kB 00:00 2023-12-11T19:19:21.868 INFO:teuthology.orchestra.run.smithi070.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-12-11T19:19:21.869 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:19:21.869 INFO:teuthology.orchestra.run.smithi070.stdout:Total 28 MB/s | 18 MB 00:00 2023-12-11T19:19:21.907 INFO:teuthology.orchestra.run.smithi167.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-12-11T19:19:21.913 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-12-11T19:19:22.021 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================================ 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================================ 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout:Installing group/module packages: 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-11T19:19:22.022 INFO:teuthology.orchestra.run.smithi066.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout:Installing module profiles: 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools/common 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling module streams: 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools rhel8 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================================ 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout:Install 10 Packages 2023-12-11T19:19:22.023 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:22.024 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 18 M 2023-12-11T19:19:22.024 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 65 M 2023-12-11T19:19:22.024 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2023-12-11T19:19:22.074 INFO:teuthology.orchestra.run.smithi167.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 20 MB/s | 7.0 MB 00:00 2023-12-11T19:19:22.101 INFO:teuthology.orchestra.run.smithi066.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 538 kB/s | 41 kB 00:00 2023-12-11T19:19:22.126 INFO:teuthology.orchestra.run.smithi066.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 245 kB/s | 25 kB 00:00 2023-12-11T19:19:22.152 INFO:teuthology.orchestra.run.smithi066.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 981 kB/s | 49 kB 00:00 2023-12-11T19:19:22.174 INFO:teuthology.orchestra.run.smithi167.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-12-11T19:19:22.175 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:19:22.175 INFO:teuthology.orchestra.run.smithi167.stdout:Total 27 MB/s | 18 MB 00:00 2023-12-11T19:19:22.177 INFO:teuthology.orchestra.run.smithi066.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 618 kB/s | 94 kB 00:00 2023-12-11T19:19:22.204 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-12-11T19:19:22.211 INFO:teuthology.orchestra.run.smithi066.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-11T19:19:22.227 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-12-11T19:19:22.227 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-12-11T19:19:22.236 INFO:teuthology.orchestra.run.smithi066.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-12-11T19:19:22.269 INFO:teuthology.orchestra.run.smithi066.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-12-11T19:19:22.386 INFO:teuthology.orchestra.run.smithi066.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-12-11T19:19:22.512 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-12-11T19:19:22.535 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-12-11T19:19:22.535 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-12-11T19:19:22.545 INFO:teuthology.orchestra.run.smithi066.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 23 MB/s | 7.0 MB 00:00 2023-12-11T19:19:22.658 INFO:teuthology.orchestra.run.smithi066.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-12-11T19:19:22.659 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:19:22.659 INFO:teuthology.orchestra.run.smithi066.stdout:Total 29 MB/s | 18 MB 00:00 2023-12-11T19:19:22.699 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-12-11T19:19:22.703 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-12-11T19:19:23.008 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2023-12-11T19:19:23.033 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2023-12-11T19:19:23.033 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2023-12-11T19:19:23.086 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-12-11T19:19:23.091 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-12-11T19:19:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-12-11T19:19:23.514 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-11T19:19:23.553 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2023-12-11T19:19:23.558 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2023-12-11T19:19:23.675 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-12-11T19:19:23.676 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T19:19:23.886 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-11T19:19:23.916 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-11T19:19:24.088 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2023-12-11T19:19:24.099 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T19:19:24.137 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-11T19:19:24.263 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-11T19:19:24.361 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-11T19:19:24.428 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T19:19:24.522 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-11T19:19:24.587 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-11T19:19:24.642 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-11T19:19:24.877 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-11T19:19:24.959 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-11T19:19:25.279 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-11T19:19:25.362 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-11T19:19:25.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-11T19:19:26.130 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-11T19:19:26.291 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-11T19:19:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-11T19:19:26.716 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-11T19:19:26.790 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-11T19:19:26.998 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-11T19:19:27.006 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T19:19:27.092 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-11T19:19:27.204 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-11T19:19:27.354 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-11T19:19:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-11T19:19:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-11T19:19:27.437 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T19:19:27.525 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-11T19:19:27.707 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-11T19:19:27.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-11T19:19:27.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-11T19:19:27.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-11T19:19:27.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-11T19:19:28.119 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-11T19:19:32.305 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-11T19:19:32.305 INFO:teuthology.orchestra.run.smithi070.stdout:Installed products updated. 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:32.529 INFO:teuthology.orchestra.run.smithi070.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T19:19:32.530 INFO:teuthology.orchestra.run.smithi070.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-11T19:19:32.530 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:19:32.530 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-12-11T19:19:32.756 DEBUG:teuthology.parallel:result is None 2023-12-11T19:19:33.677 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-11T19:19:33.677 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-12-11T19:19:33.884 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:33.884 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-12-11T19:19:33.884 INFO:teuthology.orchestra.run.smithi167.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:19:33.885 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-12-11T19:19:34.095 DEBUG:teuthology.parallel:result is None 2023-12-11T19:19:34.310 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-11T19:19:34.310 INFO:teuthology.orchestra.run.smithi066.stdout:Installed products updated. 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-11T19:19:34.530 INFO:teuthology.orchestra.run.smithi066.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:34.531 INFO:teuthology.orchestra.run.smithi066.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T19:19:34.531 INFO:teuthology.orchestra.run.smithi066.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T19:19:34.531 INFO:teuthology.orchestra.run.smithi066.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-11T19:19:34.531 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:19:34.531 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-12-11T19:19:34.737 DEBUG:teuthology.parallel:result is None 2023-12-11T19:19:34.737 INFO:teuthology.run_tasks:Running task install... 2023-12-11T19:19:34.746 DEBUG:teuthology.task.install:project ceph 2023-12-11T19:19:34.746 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d'}} 2023-12-11T19:19:34.746 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d'} 2023-12-11T19:19:34.746 INFO:teuthology.task.install:Using flavor: default 2023-12-11T19:19:34.753 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-11T19:19:34.753 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-11T19:19:34.753 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-11T19:19:34.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:19:34.756 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-11T19:19:34.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:19:34.758 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-11T19:19:34.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:19:35.001 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-11T19:19:35.002 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-11T19:19:35.006 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-11T19:19:35.006 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-11T19:19:35.011 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-11T19:19:35.011 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-11T19:19:35.113 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-11T19:19:35.113 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:19:35.114 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-11T19:19:35.175 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-11T19:19:35.175 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:19:35.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-11T19:19:35.180 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-11T19:19:35.180 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:19:35.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-11T19:19:35.185 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-11T19:19:35.185 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-11T19:19:35.250 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-11T19:19:35.250 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-11T19:19:35.256 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-11T19:19:35.256 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-11T19:19:35.297 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-11T19:19:35.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-11T19:19:35.369 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-11T19:19:35.398 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-11T19:19:35.457 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-11T19:19:35.465 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-11T19:19:35.468 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2023-12-11T19:19:35.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2023-12-11T19:19:35.525 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2023-12-11T19:19:35.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2023-12-11T19:19:35.534 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2023-12-11T19:19:35.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2023-12-11T19:19:35.956 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:35.977 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:35.987 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:37.293 INFO:teuthology.orchestra.run.smithi167.stdout:44 files removed 2023-12-11T19:19:37.293 INFO:teuthology.orchestra.run.smithi070.stdout:44 files removed 2023-12-11T19:19:37.311 INFO:teuthology.orchestra.run.smithi066.stdout:44 files removed 2023-12-11T19:19:37.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-11T19:19:37.351 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-11T19:19:37.371 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-11T19:19:37.799 INFO:teuthology.orchestra.run.smithi070.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:37.803 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:37.860 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-12-11T19:19:39.570 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 300 kB/s | 81 kB 00:00 2023-12-11T19:19:39.637 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 203 kB/s | 81 kB 00:00 2023-12-11T19:19:39.831 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 225 kB/s | 81 kB 00:00 2023-12-11T19:19:39.962 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-12-11T19:19:40.030 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-12-11T19:19:40.166 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 32 kB/s | 9.5 kB 00:00 2023-12-11T19:19:40.305 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2023-12-11T19:19:40.406 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-12-11T19:19:40.438 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-12-11T19:19:40.516 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-12-11T19:19:40.598 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-12-11T19:19:40.667 INFO:teuthology.orchestra.run.smithi066.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-12-11T19:19:41.470 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-12-11T19:19:41.665 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-12-11T19:19:44.539 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 3.9 MB/s | 16 MB 00:04 2023-12-11T19:19:45.807 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 379 kB/s | 24 kB 00:00 2023-12-11T19:19:46.026 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 128 kB/s | 24 kB 00:00 2023-12-11T19:19:47.301 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 MB/s | 53 MB 00:01 2023-12-11T19:19:47.470 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 MB/s | 47 MB 00:01 2023-12-11T19:19:48.730 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 422 kB/s | 24 kB 00:00 2023-12-11T19:19:49.990 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-12-11T19:19:56.228 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 MB/s | 47 MB 00:01 2023-12-11T19:19:56.770 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 MB/s | 53 MB 00:01 2023-12-11T19:19:58.774 INFO:teuthology.orchestra.run.smithi070.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 47 MB 00:01 2023-12-11T19:20:07.897 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T19:20:07.899 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T19:20:07.917 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-11T19:20:07.917 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-11T19:20:07.917 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-11T19:20:07.918 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-11T19:20:07.920 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-11T19:20:07.924 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-11T19:20:07.925 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-11T19:20:08.067 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-11T19:20:08.080 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-11T19:20:08.081 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-11T19:20:08.082 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-11T19:20:08.083 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-11T19:20:08.084 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-11T19:20:08.085 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-11T19:20:08.086 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-11T19:20:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-11T19:20:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-11T19:20:08.089 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql 10 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout:Install 129 Packages 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 3 Packages 2023-12-11T19:20:08.090 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:20:08.091 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 209 M 2023-12-11T19:20:08.091 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-12-11T19:20:08.564 INFO:teuthology.orchestra.run.smithi167.stdout:(1/132): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 48 kB/s | 6.4 kB 00:00 2023-12-11T19:20:08.914 INFO:teuthology.orchestra.run.smithi167.stdout:(2/132): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 2.6 MB/s | 919 kB 00:00 2023-12-11T19:20:09.006 INFO:teuthology.orchestra.run.smithi167.stdout:(3/132): ceph-immutable-object-cache-19.0.0-119 1.8 MB/s | 171 kB 00:00 2023-12-11T19:20:09.123 INFO:teuthology.orchestra.run.smithi167.stdout:(4/132): ceph-base-19.0.0-119.ga62fda73.el8.x86 7.8 MB/s | 5.4 MB 00:00 2023-12-11T19:20:09.315 INFO:teuthology.orchestra.run.smithi167.stdout:(5/132): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 8.5 MB/s | 1.6 MB 00:00 2023-12-11T19:20:09.390 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T19:20:09.392 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T19:20:09.407 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-11T19:20:09.408 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-11T19:20:09.409 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-11T19:20:09.411 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-11T19:20:09.413 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-11T19:20:09.417 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-11T19:20:09.418 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-11T19:20:09.567 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-12-11T19:20:09.581 INFO:teuthology.orchestra.run.smithi066.stdout:=========================================================================================================================== 2023-12-11T19:20:09.581 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2023-12-11T19:20:09.581 INFO:teuthology.orchestra.run.smithi066.stdout:=========================================================================================================================== 2023-12-11T19:20:09.581 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-11T19:20:09.582 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-11T19:20:09.583 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-11T19:20:09.584 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-11T19:20:09.585 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-11T19:20:09.586 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-11T19:20:09.587 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-11T19:20:09.588 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T19:20:09.589 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-11T19:20:09.590 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling module streams: 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql 10 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout:=========================================================================================================================== 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout:Install 129 Packages 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 3 Packages 2023-12-11T19:20:09.591 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:20:09.592 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 209 M 2023-12-11T19:20:09.592 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2023-12-11T19:20:09.740 INFO:teuthology.orchestra.run.smithi167.stdout:(6/132): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 3.1 MB/s | 2.3 MB 00:00 2023-12-11T19:20:09.815 INFO:teuthology.orchestra.run.smithi167.stdout:(7/132): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 9.3 MB/s | 4.6 MB 00:00 2023-12-11T19:20:09.988 INFO:teuthology.orchestra.run.smithi066.stdout:(1/132): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 27 kB/s | 6.4 kB 00:00 2023-12-11T19:20:10.355 INFO:teuthology.orchestra.run.smithi066.stdout:(2/132): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 2.4 MB/s | 919 kB 00:00 2023-12-11T19:20:10.416 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T19:20:10.417 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T19:20:10.422 INFO:teuthology.orchestra.run.smithi066.stdout:(3/132): ceph-immutable-object-cache-19.0.0-119 2.5 MB/s | 171 kB 00:00 2023-12-11T19:20:10.432 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-11T19:20:10.433 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-11T19:20:10.435 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-11T19:20:10.436 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-11T19:20:10.438 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-11T19:20:10.441 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-11T19:20:10.442 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-11T19:20:10.497 INFO:teuthology.orchestra.run.smithi167.stdout:(8/132): ceph-common-19.0.0-119.ga62fda73.el8.x 11 MB/s | 23 MB 00:02 2023-12-11T19:20:10.549 INFO:teuthology.orchestra.run.smithi167.stdout:(9/132): ceph-selinux-19.0.0-119.ga62fda73.el8. 473 kB/s | 24 kB 00:00 2023-12-11T19:20:10.579 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-12-11T19:20:10.591 INFO:teuthology.orchestra.run.smithi070.stdout:=========================================================================================================================== 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout:=========================================================================================================================== 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-11T19:20:10.592 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-11T19:20:10.593 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-11T19:20:10.594 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-11T19:20:10.595 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-11T19:20:10.596 INFO:teuthology.orchestra.run.smithi070.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-11T19:20:10.597 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-11T19:20:10.598 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-11T19:20:10.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-11T19:20:10.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T19:20:10.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-11T19:20:10.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-11T19:20:10.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-11T19:20:10.653 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-11T19:20:10.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-11T19:20:10.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-11T19:20:10.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-11T19:20:10.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-11T19:20:10.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-11T19:20:10.654 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-11T19:20:10.655 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-11T19:20:10.656 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling module streams: 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql 10 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:=========================================================================================================================== 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:Install 129 Packages 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 3 Packages 2023-12-11T19:20:10.657 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:20:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 209 M 2023-12-11T19:20:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-12-11T19:20:11.039 INFO:teuthology.orchestra.run.smithi066.stdout:(4/132): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 3.7 MB/s | 2.3 MB 00:00 2023-12-11T19:20:11.132 INFO:teuthology.orchestra.run.smithi070.stdout:(1/132): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 18 kB/s | 6.4 kB 00:00 2023-12-11T19:20:11.475 INFO:teuthology.orchestra.run.smithi167.stdout:(10/132): ceph-radosgw-19.0.0-119.ga62fda73.el8 8.4 MB/s | 14 MB 00:01 2023-12-11T19:20:11.572 INFO:teuthology.orchestra.run.smithi066.stdout:(5/132): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 3.1 MB/s | 1.6 MB 00:00 2023-12-11T19:20:11.574 INFO:teuthology.orchestra.run.smithi167.stdout:(11/132): cephfs-mirror-19.0.0-119.ga62fda73.el 2.6 MB/s | 262 kB 00:00 2023-12-11T19:20:11.667 INFO:teuthology.orchestra.run.smithi167.stdout:(12/132): libcephfs-devel-19.0.0-119.ga62fda73. 339 kB/s | 31 kB 00:00 2023-12-11T19:20:11.909 INFO:teuthology.orchestra.run.smithi167.stdout:(13/132): libcephfs2-19.0.0-119.ga62fda73.el8.x 3.4 MB/s | 838 kB 00:00 2023-12-11T19:20:11.932 INFO:teuthology.orchestra.run.smithi070.stdout:(2/132): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 1.1 MB/s | 919 kB 00:00 2023-12-11T19:20:11.984 INFO:teuthology.orchestra.run.smithi167.stdout:(14/132): libcephsqlite-19.0.0-119.ga62fda73.el 2.4 MB/s | 183 kB 00:00 2023-12-11T19:20:12.067 INFO:teuthology.orchestra.run.smithi167.stdout:(15/132): librados-devel-19.0.0-119.ga62fda73.e 1.6 MB/s | 139 kB 00:00 2023-12-11T19:20:12.099 INFO:teuthology.orchestra.run.smithi070.stdout:(3/132): ceph-immutable-object-cache-19.0.0-119 1.0 MB/s | 171 kB 00:00 2023-12-11T19:20:12.301 INFO:teuthology.orchestra.run.smithi167.stdout:(16/132): libradosstriper1-19.0.0-119.ga62fda73 2.2 MB/s | 517 kB 00:00 2023-12-11T19:20:12.341 INFO:teuthology.orchestra.run.smithi070.stdout:(4/132): ceph-base-19.0.0-119.ga62fda73.el8.x86 3.4 MB/s | 5.4 MB 00:01 2023-12-11T19:20:12.581 INFO:teuthology.orchestra.run.smithi066.stdout:(6/132): ceph-base-19.0.0-119.ga62fda73.el8.x86 1.9 MB/s | 5.4 MB 00:02 2023-12-11T19:20:12.716 INFO:teuthology.orchestra.run.smithi070.stdout:(5/132): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 4.3 MB/s | 1.6 MB 00:00 2023-12-11T19:20:13.000 INFO:teuthology.orchestra.run.smithi070.stdout:(6/132): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 2.6 MB/s | 2.3 MB 00:00 2023-12-11T19:20:13.625 INFO:teuthology.orchestra.run.smithi070.stdout:(7/132): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 5.1 MB/s | 4.6 MB 00:00 2023-12-11T19:20:13.739 INFO:teuthology.orchestra.run.smithi066.stdout:(7/132): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 2.1 MB/s | 4.6 MB 00:02 2023-12-11T19:20:14.518 INFO:teuthology.orchestra.run.smithi167.stdout:(17/132): librgw2-19.0.0-119.ga62fda73.el8.x86_ 3.1 MB/s | 6.9 MB 00:02 2023-12-11T19:20:14.610 INFO:teuthology.orchestra.run.smithi167.stdout:(18/132): python3-ceph-argparse-19.0.0-119.ga62 489 kB/s | 45 kB 00:00 2023-12-11T19:20:14.710 INFO:teuthology.orchestra.run.smithi167.stdout:(19/132): python3-ceph-common-19.0.0-119.ga62fd 1.3 MB/s | 129 kB 00:00 2023-12-11T19:20:14.819 INFO:teuthology.orchestra.run.smithi167.stdout:(20/132): python3-cephfs-19.0.0-119.ga62fda73.e 1.8 MB/s | 197 kB 00:00 2023-12-11T19:20:15.003 INFO:teuthology.orchestra.run.smithi167.stdout:(21/132): python3-rados-19.0.0-119.ga62fda73.el 2.0 MB/s | 371 kB 00:00 2023-12-11T19:20:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:(22/132): python3-rbd-19.0.0-119.ga62fda73.el8. 2.0 MB/s | 364 kB 00:00 2023-12-11T19:20:15.278 INFO:teuthology.orchestra.run.smithi167.stdout:(23/132): python3-rgw-19.0.0-119.ga62fda73.el8. 1.0 MB/s | 107 kB 00:00 2023-12-11T19:20:15.453 INFO:teuthology.orchestra.run.smithi167.stdout:(24/132): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 533 kB/s | 93 kB 00:00 2023-12-11T19:20:16.100 INFO:teuthology.orchestra.run.smithi070.stdout:(8/132): ceph-osd-19.0.0-119.ga62fda73.el8.x86_ 5.8 MB/s | 18 MB 00:03 2023-12-11T19:20:16.200 INFO:teuthology.orchestra.run.smithi070.stdout:(9/132): ceph-selinux-19.0.0-119.ga62fda73.el8. 244 kB/s | 24 kB 00:00 2023-12-11T19:20:16.545 INFO:teuthology.orchestra.run.smithi167.stdout:(25/132): rbd-mirror-19.0.0-119.ga62fda73.el8.x 3.3 MB/s | 3.6 MB 00:01 2023-12-11T19:20:16.629 INFO:teuthology.orchestra.run.smithi167.stdout:(26/132): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 2.1 MB/s | 181 kB 00:00 2023-12-11T19:20:16.642 INFO:teuthology.orchestra.run.smithi070.stdout:(10/132): ceph-radosgw-19.0.0-119.ga62fda73.el8 4.6 MB/s | 14 MB 00:03 2023-12-11T19:20:16.712 INFO:teuthology.orchestra.run.smithi167.stdout:(27/132): ceph-grafana-dashboards-19.0.0-119.ga 338 kB/s | 28 kB 00:00 2023-12-11T19:20:16.759 INFO:teuthology.orchestra.run.smithi070.stdout:(11/132): cephfs-mirror-19.0.0-119.ga62fda73.el 2.2 MB/s | 262 kB 00:00 2023-12-11T19:20:16.868 INFO:teuthology.orchestra.run.smithi070.stdout:(12/132): libcephfs-devel-19.0.0-119.ga62fda73. 286 kB/s | 31 kB 00:00 2023-12-11T19:20:16.954 INFO:teuthology.orchestra.run.smithi167.stdout:(28/132): ceph-osd-19.0.0-119.ga62fda73.el8.x86 2.5 MB/s | 18 MB 00:07 2023-12-11T19:20:16.988 INFO:teuthology.orchestra.run.smithi167.stdout:(29/132): ceph-mgr-cephadm-19.0.0-119.ga62fda73 481 kB/s | 133 kB 00:00 2023-12-11T19:20:17.043 INFO:teuthology.orchestra.run.smithi070.stdout:(13/132): libcephfs2-19.0.0-119.ga62fda73.el8.x 4.7 MB/s | 838 kB 00:00 2023-12-11T19:20:17.143 INFO:teuthology.orchestra.run.smithi070.stdout:(14/132): libcephsqlite-19.0.0-119.ga62fda73.el 1.8 MB/s | 183 kB 00:00 2023-12-11T19:20:17.252 INFO:teuthology.orchestra.run.smithi070.stdout:(15/132): librados-devel-19.0.0-119.ga62fda73.e 1.3 MB/s | 139 kB 00:00 2023-12-11T19:20:17.544 INFO:teuthology.orchestra.run.smithi070.stdout:(16/132): libradosstriper1-19.0.0-119.ga62fda73 1.7 MB/s | 517 kB 00:00 2023-12-11T19:20:17.655 INFO:teuthology.orchestra.run.smithi167.stdout:(30/132): ceph-mgr-dashboard-19.0.0-119.ga62fda 2.5 MB/s | 1.8 MB 00:00 2023-12-11T19:20:18.251 INFO:teuthology.orchestra.run.smithi167.stdout:(31/132): ceph-test-19.0.0-119.ga62fda73.el8.x8 6.4 MB/s | 49 MB 00:07 2023-12-11T19:20:18.289 INFO:teuthology.orchestra.run.smithi167.stdout:(32/132): ceph-mgr-modules-core-19.0.0-119.ga62 384 kB/s | 243 kB 00:00 2023-12-11T19:20:18.323 INFO:teuthology.orchestra.run.smithi167.stdout:(33/132): ceph-mgr-rook-19.0.0-119.ga62fda73.el 706 kB/s | 50 kB 00:00 2023-12-11T19:20:18.365 INFO:teuthology.orchestra.run.smithi167.stdout:(34/132): ceph-prometheus-alerts-19.0.0-119.ga6 199 kB/s | 15 kB 00:00 2023-12-11T19:20:18.457 INFO:teuthology.orchestra.run.smithi167.stdout:(35/132): ceph-volume-19.0.0-119.ga62fda73.el8. 1.9 MB/s | 262 kB 00:00 2023-12-11T19:20:18.532 INFO:teuthology.orchestra.run.smithi167.stdout:(36/132): cephfs-top-19.0.0-119.ga62fda73.el8.n 242 kB/s | 18 kB 00:00 2023-12-11T19:20:18.607 INFO:teuthology.orchestra.run.smithi167.stdout:(37/132): cephadm-19.0.0-119.ga62fda73.el8.noar 1.7 MB/s | 415 kB 00:00 2023-12-11T19:20:18.877 INFO:teuthology.orchestra.run.smithi070.stdout:(17/132): ceph-common-19.0.0-119.ga62fda73.el8. 2.8 MB/s | 23 MB 00:08 2023-12-11T19:20:18.936 INFO:teuthology.orchestra.run.smithi070.stdout:(18/132): python3-ceph-argparse-19.0.0-119.ga62 770 kB/s | 45 kB 00:00 2023-12-11T19:20:19.028 INFO:teuthology.orchestra.run.smithi070.stdout:(19/132): librgw2-19.0.0-119.ga62fda73.el8.x86_ 4.7 MB/s | 6.9 MB 00:01 2023-12-11T19:20:19.053 INFO:teuthology.orchestra.run.smithi070.stdout:(20/132): python3-ceph-common-19.0.0-119.ga62fd 1.1 MB/s | 129 kB 00:00 2023-12-11T19:20:19.111 INFO:teuthology.orchestra.run.smithi070.stdout:(21/132): python3-cephfs-19.0.0-119.ga62fda73.e 2.3 MB/s | 197 kB 00:00 2023-12-11T19:20:19.174 INFO:teuthology.orchestra.run.smithi167.stdout:(38/132): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2023-12-11T19:20:19.199 INFO:teuthology.orchestra.run.smithi167.stdout:(39/132): liboath-2.6.2-3.el8.x86_64.rpm 99 kB/s | 59 kB 00:00 2023-12-11T19:20:19.204 INFO:teuthology.orchestra.run.smithi070.stdout:(22/132): python3-rados-19.0.0-119.ga62fda73.el 2.4 MB/s | 371 kB 00:00 2023-12-11T19:20:19.229 INFO:teuthology.orchestra.run.smithi070.stdout:(23/132): python3-rbd-19.0.0-119.ga62fda73.el8. 3.0 MB/s | 364 kB 00:00 2023-12-11T19:20:19.279 INFO:teuthology.orchestra.run.smithi070.stdout:(24/132): python3-rgw-19.0.0-119.ga62fda73.el8. 1.4 MB/s | 107 kB 00:00 2023-12-11T19:20:19.305 INFO:teuthology.orchestra.run.smithi070.stdout:(25/132): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 1.2 MB/s | 93 kB 00:00 2023-12-11T19:20:19.388 INFO:teuthology.orchestra.run.smithi070.stdout:(26/132): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 2.1 MB/s | 181 kB 00:00 2023-12-11T19:20:19.425 INFO:teuthology.orchestra.run.smithi167.stdout:(40/132): gperftools-libs-2.7-9.el8.x86_64.rpm 343 kB/s | 306 kB 00:00 2023-12-11T19:20:19.464 INFO:teuthology.orchestra.run.smithi070.stdout:(27/132): ceph-grafana-dashboards-19.0.0-119.ga 373 kB/s | 28 kB 00:00 2023-12-11T19:20:19.483 INFO:teuthology.orchestra.run.smithi167.stdout:(41/132): luarocks-3.5.0-1.el8.noarch.rpm 589 kB/s | 167 kB 00:00 2023-12-11T19:20:19.539 INFO:teuthology.orchestra.run.smithi070.stdout:(28/132): ceph-mgr-cephadm-19.0.0-119.ga62fda73 1.7 MB/s | 133 kB 00:00 2023-12-11T19:20:19.592 INFO:teuthology.orchestra.run.smithi167.stdout:(42/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 404 kB/s | 44 kB 00:00 2023-12-11T19:20:19.617 INFO:teuthology.orchestra.run.smithi167.stdout:(43/132): libunwind-1.3.1-3.el8.x86_64.rpm 170 kB/s | 75 kB 00:00 2023-12-11T19:20:19.701 INFO:teuthology.orchestra.run.smithi167.stdout:(44/132): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-12-11T19:20:19.754 INFO:teuthology.orchestra.run.smithi167.stdout:(45/132): python3-cachetools-3.1.1-4.el8.noarch 241 kB/s | 33 kB 00:00 2023-12-11T19:20:19.810 INFO:teuthology.orchestra.run.smithi167.stdout:(46/132): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-12-11T19:20:19.943 INFO:teuthology.orchestra.run.smithi167.stdout:(47/132): python3-cheroot-8.5.2-1.el8.noarch.rp 916 kB/s | 173 kB 00:00 2023-12-11T19:20:19.947 INFO:teuthology.orchestra.run.smithi070.stdout:(29/132): ceph-mgr-dashboard-19.0.0-119.ga62fda 4.3 MB/s | 1.8 MB 00:00 2023-12-11T19:20:19.994 INFO:teuthology.orchestra.run.smithi167.stdout:(48/132): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-12-11T19:20:20.019 INFO:teuthology.orchestra.run.smithi167.stdout:(49/132): python3-asyncssh-2.7.0-2.el8.noarch.r 801 kB/s | 476 kB 00:00 2023-12-11T19:20:20.044 INFO:teuthology.orchestra.run.smithi167.stdout:(50/132): python3-google-auth-1.1.1-10.el8.noar 808 kB/s | 81 kB 00:00 2023-12-11T19:20:20.081 INFO:teuthology.orchestra.run.smithi070.stdout:(30/132): rbd-mirror-19.0.0-119.ga62fda73.el8.x 4.5 MB/s | 3.6 MB 00:00 2023-12-11T19:20:20.094 INFO:teuthology.orchestra.run.smithi167.stdout:(51/132): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2023-12-11T19:20:20.120 INFO:teuthology.orchestra.run.smithi167.stdout:(52/132): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-12-11T19:20:20.173 INFO:teuthology.orchestra.run.smithi070.stdout:(31/132): ceph-mgr-modules-core-19.0.0-119.ga62 2.6 MB/s | 243 kB 00:00 2023-12-11T19:20:20.195 INFO:teuthology.orchestra.run.smithi167.stdout:(53/132): python3-logutils-0.3.5-11.el8.noarch. 486 kB/s | 49 kB 00:00 2023-12-11T19:20:20.220 INFO:teuthology.orchestra.run.smithi167.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 586 kB/s | 59 kB 00:00 2023-12-11T19:20:20.231 INFO:teuthology.orchestra.run.smithi066.stdout:(8/132): ceph-radosgw-19.0.0-119.ga62fda73.el8. 2.1 MB/s | 14 MB 00:06 2023-12-11T19:20:20.232 INFO:teuthology.orchestra.run.smithi070.stdout:(32/132): ceph-mgr-rook-19.0.0-119.ga62fda73.el 857 kB/s | 50 kB 00:00 2023-12-11T19:20:20.290 INFO:teuthology.orchestra.run.smithi070.stdout:(33/132): ceph-prometheus-alerts-19.0.0-119.ga6 258 kB/s | 15 kB 00:00 2023-12-11T19:20:20.297 INFO:teuthology.orchestra.run.smithi066.stdout:(9/132): ceph-selinux-19.0.0-119.ga62fda73.el8. 367 kB/s | 24 kB 00:00 2023-12-11T19:20:20.304 INFO:teuthology.orchestra.run.smithi167.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-12-11T19:20:20.346 INFO:teuthology.orchestra.run.smithi167.stdout:(56/132): python3-kubernetes-11.0.0-6.el8.noarc 3.9 MB/s | 1.2 MB 00:00 2023-12-11T19:20:20.382 INFO:teuthology.orchestra.run.smithi070.stdout:(34/132): ceph-volume-19.0.0-119.ga62fda73.el8. 2.8 MB/s | 262 kB 00:00 2023-12-11T19:20:20.397 INFO:teuthology.orchestra.run.smithi167.stdout:(57/132): python3-portend-2.6-1.el8.noarch.rpm 169 kB/s | 16 kB 00:00 2023-12-11T19:20:20.499 INFO:teuthology.orchestra.run.smithi070.stdout:(35/132): cephadm-19.0.0-119.ga62fda73.el8.noar 3.5 MB/s | 415 kB 00:00 2023-12-11T19:20:20.522 INFO:teuthology.orchestra.run.smithi167.stdout:(58/132): python3-repoze-lru-0.7-6.el8.noarch.r 192 kB/s | 34 kB 00:00 2023-12-11T19:20:20.547 INFO:teuthology.orchestra.run.smithi167.stdout:(59/132): python3-pecan-1.3.2-9.el8.noarch.rpm 866 kB/s | 283 kB 00:00 2023-12-11T19:20:20.572 INFO:teuthology.orchestra.run.smithi167.stdout:(60/132): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-12-11T19:20:20.623 INFO:teuthology.orchestra.run.smithi167.stdout:(61/132): python3-rsa-4.9-2.el8.noarch.rpm 639 kB/s | 64 kB 00:00 2023-12-11T19:20:20.648 INFO:teuthology.orchestra.run.smithi167.stdout:(62/132): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-12-11T19:20:20.665 INFO:teuthology.orchestra.run.smithi066.stdout:(10/132): ceph-common-19.0.0-119.ga62fda73.el8. 2.1 MB/s | 23 MB 00:10 2023-12-11T19:20:20.673 INFO:teuthology.orchestra.run.smithi167.stdout:(63/132): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-12-11T19:20:20.723 INFO:teuthology.orchestra.run.smithi167.stdout:(64/132): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-12-11T19:20:20.749 INFO:teuthology.orchestra.run.smithi070.stdout:(36/132): cephfs-top-19.0.0-119.ga62fda73.el8.n 72 kB/s | 18 kB 00:00 2023-12-11T19:20:20.764 INFO:teuthology.orchestra.run.smithi066.stdout:(11/132): cephfs-mirror-19.0.0-119.ga62fda73.el 2.6 MB/s | 262 kB 00:00 2023-12-11T19:20:20.782 INFO:teuthology.orchestra.run.smithi167.stdout:(65/132): python3-trustme-0.6.0-4.el8.noarch.rp 206 kB/s | 27 kB 00:00 2023-12-11T19:20:20.808 INFO:teuthology.orchestra.run.smithi167.stdout:(66/132): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-12-11T19:20:20.833 INFO:teuthology.orchestra.run.smithi167.stdout:(67/132): python3-webob-1.8.5-1.el8.1.noarch.rp 2.2 MB/s | 251 kB 00:00 2023-12-11T19:20:20.883 INFO:teuthology.orchestra.run.smithi167.stdout:(68/132): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2023-12-11T19:20:20.909 INFO:teuthology.orchestra.run.smithi167.stdout:(69/132): python3-webtest-2.0.33-1.el8.noarch.r 849 kB/s | 85 kB 00:00 2023-12-11T19:20:20.934 INFO:teuthology.orchestra.run.smithi167.stdout:(70/132): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-12-11T19:20:20.957 INFO:teuthology.orchestra.run.smithi066.stdout:(12/132): ceph-osd-19.0.0-119.ga62fda73.el8.x86 2.2 MB/s | 18 MB 00:08 2023-12-11T19:20:20.959 INFO:teuthology.orchestra.run.smithi167.stdout:(71/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-11T19:20:20.982 INFO:teuthology.orchestra.run.smithi066.stdout:(13/132): libcephfs-devel-19.0.0-119.ga62fda73. 143 kB/s | 31 kB 00:00 2023-12-11T19:20:21.026 INFO:teuthology.orchestra.run.smithi167.stdout:(72/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-12-11T19:20:21.051 INFO:teuthology.orchestra.run.smithi167.stdout:(73/132): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-12-11T19:20:21.073 INFO:teuthology.orchestra.run.smithi066.stdout:(14/132): libcephsqlite-19.0.0-119.ga62fda73.el 2.0 MB/s | 183 kB 00:00 2023-12-11T19:20:21.076 INFO:teuthology.orchestra.run.smithi167.stdout:(74/132): python3-jwt-1.6.1-2.el8.noarch.rpm 855 kB/s | 43 kB 00:00 2023-12-11T19:20:21.110 INFO:teuthology.orchestra.run.smithi167.stdout:(75/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-12-11T19:20:21.124 INFO:teuthology.orchestra.run.smithi066.stdout:(15/132): librados-devel-19.0.0-119.ga62fda73.e 2.7 MB/s | 139 kB 00:00 2023-12-11T19:20:21.135 INFO:teuthology.orchestra.run.smithi167.stdout:(76/132): keyutils-libs-devel-1.5.10-9.el8.x86_ 825 kB/s | 48 kB 00:00 2023-12-11T19:20:21.142 INFO:teuthology.orchestra.run.smithi070.stdout:(37/132): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2023-12-11T19:20:21.161 INFO:teuthology.orchestra.run.smithi167.stdout:(77/132): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-12-11T19:20:21.174 INFO:teuthology.orchestra.run.smithi066.stdout:(16/132): libcephfs2-19.0.0-119.ga62fda73.el8.x 3.8 MB/s | 838 kB 00:00 2023-12-11T19:20:21.186 INFO:teuthology.orchestra.run.smithi167.stdout:(78/132): python3-requests-oauthlib-1.0.0-1.el8 850 kB/s | 43 kB 00:00 2023-12-11T19:20:21.211 INFO:teuthology.orchestra.run.smithi167.stdout:(79/132): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-12-11T19:20:21.258 INFO:teuthology.orchestra.run.smithi066.stdout:(17/132): libradosstriper1-19.0.0-119.ga62fda73 3.8 MB/s | 517 kB 00:00 2023-12-11T19:20:21.267 INFO:teuthology.orchestra.run.smithi167.stdout:(80/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.8 MB/s | 229 kB 00:00 2023-12-11T19:20:21.308 INFO:teuthology.orchestra.run.smithi066.stdout:(18/132): python3-ceph-argparse-19.0.0-119.ga62 903 kB/s | 45 kB 00:00 2023-12-11T19:20:21.312 INFO:teuthology.orchestra.run.smithi167.stdout:(81/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.5 MB/s | 560 kB 00:00 2023-12-11T19:20:21.337 INFO:teuthology.orchestra.run.smithi167.stdout:(82/132): libaio-devel-0.3.112-1.el8.x86_64.rpm 267 kB/s | 19 kB 00:00 2023-12-11T19:20:21.358 INFO:teuthology.orchestra.run.smithi066.stdout:(19/132): python3-ceph-common-19.0.0-119.ga62fd 2.5 MB/s | 129 kB 00:00 2023-12-11T19:20:21.362 INFO:teuthology.orchestra.run.smithi167.stdout:(83/132): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-12-11T19:20:21.397 INFO:teuthology.orchestra.run.smithi167.stdout:(84/132): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-12-11T19:20:21.422 INFO:teuthology.orchestra.run.smithi167.stdout:(85/132): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-12-11T19:20:21.434 INFO:teuthology.orchestra.run.smithi066.stdout:(20/132): python3-cephfs-19.0.0-119.ga62fda73.e 2.6 MB/s | 197 kB 00:00 2023-12-11T19:20:21.455 INFO:teuthology.orchestra.run.smithi167.stdout:(86/132): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-12-11T19:20:21.481 INFO:teuthology.orchestra.run.smithi167.stdout:(87/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-12-11T19:20:21.522 INFO:teuthology.orchestra.run.smithi167.stdout:(88/132): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-12-11T19:20:21.534 INFO:teuthology.orchestra.run.smithi066.stdout:(21/132): python3-rados-19.0.0-119.ga62fda73.el 3.6 MB/s | 371 kB 00:00 2023-12-11T19:20:21.556 INFO:teuthology.orchestra.run.smithi167.stdout:(89/132): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-12-11T19:20:21.607 INFO:teuthology.orchestra.run.smithi167.stdout:(90/132): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-12-11T19:20:21.632 INFO:teuthology.orchestra.run.smithi167.stdout:(91/132): elfutils-libelf-devel-0.186-1.el8.x86 797 kB/s | 60 kB 00:00 2023-12-11T19:20:21.635 INFO:teuthology.orchestra.run.smithi066.stdout:(22/132): python3-rbd-19.0.0-119.ga62fda73.el8. 3.6 MB/s | 364 kB 00:00 2023-12-11T19:20:21.657 INFO:teuthology.orchestra.run.smithi167.stdout:(92/132): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-12-11T19:20:21.676 INFO:teuthology.orchestra.run.smithi066.stdout:(23/132): python3-rgw-19.0.0-119.ga62fda73.el8. 2.5 MB/s | 107 kB 00:00 2023-12-11T19:20:21.682 INFO:teuthology.orchestra.run.smithi167.stdout:(93/132): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-12-11T19:20:21.708 INFO:teuthology.orchestra.run.smithi167.stdout:(94/132): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-12-11T19:20:21.726 INFO:teuthology.orchestra.run.smithi066.stdout:(24/132): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 1.8 MB/s | 93 kB 00:00 2023-12-11T19:20:21.770 INFO:teuthology.orchestra.run.smithi167.stdout:(95/132): libgfortran-8.5.0-10.1.el8_6.x86_64.r 7.2 MB/s | 644 kB 00:00 2023-12-11T19:20:21.775 INFO:teuthology.orchestra.run.smithi070.stdout:(38/132): ceph-test-19.0.0-119.ga62fda73.el8.x8 8.9 MB/s | 49 MB 00:05 2023-12-11T19:20:21.800 INFO:teuthology.orchestra.run.smithi167.stdout:(96/132): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.8 MB/s | 171 kB 00:00 2023-12-11T19:20:21.800 INFO:teuthology.orchestra.run.smithi070.stdout:(39/132): liboath-2.6.2-3.el8.x86_64.rpm 89 kB/s | 59 kB 00:00 2023-12-11T19:20:21.825 INFO:teuthology.orchestra.run.smithi167.stdout:(97/132): NetworkManager-initscripts-updown-1.3 2.5 MB/s | 138 kB 00:00 2023-12-11T19:20:21.867 INFO:teuthology.orchestra.run.smithi167.stdout:(98/132): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-12-11T19:20:21.901 INFO:teuthology.orchestra.run.smithi167.stdout:(99/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-11T19:20:21.934 INFO:teuthology.orchestra.run.smithi070.stdout:(40/132): gperftools-libs-2.7-9.el8.x86_64.rpm 259 kB/s | 306 kB 00:01 2023-12-11T19:20:21.943 INFO:teuthology.orchestra.run.smithi167.stdout:(100/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-12-11T19:20:21.985 INFO:teuthology.orchestra.run.smithi167.stdout:(101/132): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-12-11T19:20:22.018 INFO:teuthology.orchestra.run.smithi167.stdout:(102/132): python3-markupsafe-0.23-19.el8.x86_6 1.2 MB/s | 39 kB 00:00 2023-12-11T19:20:22.069 INFO:teuthology.orchestra.run.smithi167.stdout:(103/132): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2023-12-11T19:20:22.101 INFO:teuthology.orchestra.run.smithi070.stdout:(41/132): luarocks-3.5.0-1.el8.noarch.rpm 556 kB/s | 167 kB 00:00 2023-12-11T19:20:22.127 INFO:teuthology.orchestra.run.smithi167.stdout:(104/132): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2023-12-11T19:20:22.169 INFO:teuthology.orchestra.run.smithi167.stdout:(105/132): python3-jinja2-2.10.1-3.el8.noarch.r 5.3 MB/s | 538 kB 00:00 2023-12-11T19:20:22.186 INFO:teuthology.orchestra.run.smithi167.stdout:(106/132): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-12-11T19:20:22.201 INFO:teuthology.orchestra.run.smithi070.stdout:(42/132): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-12-11T19:20:22.211 INFO:teuthology.orchestra.run.smithi167.stdout:(107/132): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-12-11T19:20:22.227 INFO:teuthology.orchestra.run.smithi070.stdout:(43/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-12-11T19:20:22.269 INFO:teuthology.orchestra.run.smithi070.stdout:(44/132): libunwind-1.3.1-3.el8.x86_64.rpm 152 kB/s | 75 kB 00:00 2023-12-11T19:20:22.274 INFO:teuthology.orchestra.run.smithi167.stdout:(108/132): automake-1.16.1-7.el8.noarch.rpm 8.0 MB/s | 713 kB 00:00 2023-12-11T19:20:22.277 INFO:teuthology.orchestra.run.smithi066.stdout:(25/132): librgw2-19.0.0-119.ga62fda73.el8.x86_ 6.3 MB/s | 6.9 MB 00:01 2023-12-11T19:20:22.320 INFO:teuthology.orchestra.run.smithi167.stdout:(109/132): python3-jsonpatch-1.21-2.el8.noarch. 579 kB/s | 27 kB 00:00 2023-12-11T19:20:22.327 INFO:teuthology.orchestra.run.smithi070.stdout:(45/132): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-12-11T19:20:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:(46/132): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2023-12-11T19:20:22.362 INFO:teuthology.orchestra.run.smithi167.stdout:(110/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-11T19:20:22.369 INFO:teuthology.orchestra.run.smithi066.stdout:(26/132): rbd-mirror-19.0.0-119.ga62fda73.el8.x 5.6 MB/s | 3.6 MB 00:00 2023-12-11T19:20:22.378 INFO:teuthology.orchestra.run.smithi070.stdout:(47/132): python3-certifi-2018.10.15-7.el8.noar 149 kB/s | 16 kB 00:00 2023-12-11T19:20:22.394 INFO:teuthology.orchestra.run.smithi066.stdout:(27/132): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 1.5 MB/s | 181 kB 00:00 2023-12-11T19:20:22.396 INFO:teuthology.orchestra.run.smithi167.stdout:(111/132): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-12-11T19:20:22.411 INFO:teuthology.orchestra.run.smithi066.stdout:(28/132): ceph-grafana-dashboards-19.0.0-119.ga 672 kB/s | 28 kB 00:00 2023-12-11T19:20:22.445 INFO:teuthology.orchestra.run.smithi066.stdout:(29/132): ceph-mgr-cephadm-19.0.0-119.ga62fda73 2.6 MB/s | 133 kB 00:00 2023-12-11T19:20:22.471 INFO:teuthology.orchestra.run.smithi070.stdout:(48/132): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-12-11T19:20:22.513 INFO:teuthology.orchestra.run.smithi167.stdout:(112/132): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-12-11T19:20:22.545 INFO:teuthology.orchestra.run.smithi070.stdout:(49/132): python3-google-auth-1.1.1-10.el8.noar 486 kB/s | 81 kB 00:00 2023-12-11T19:20:22.570 INFO:teuthology.orchestra.run.smithi070.stdout:(50/132): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2023-12-11T19:20:22.587 INFO:teuthology.orchestra.run.smithi070.stdout:(51/132): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-12-11T19:20:22.646 INFO:teuthology.orchestra.run.smithi070.stdout:(52/132): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-12-11T19:20:22.689 INFO:teuthology.orchestra.run.smithi167.stdout:(113/132): python3-scipy-1.0.0-21.module+el8.5. 15 MB/s | 14 MB 00:00 2023-12-11T19:20:22.695 INFO:teuthology.orchestra.run.smithi066.stdout:(30/132): ceph-mgr-dashboard-19.0.0-119.ga62fda 6.2 MB/s | 1.8 MB 00:00 2023-12-11T19:20:22.696 INFO:teuthology.orchestra.run.smithi070.stdout:(53/132): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-12-11T19:20:22.714 INFO:teuthology.orchestra.run.smithi167.stdout:(114/132): python3-werkzeug-0.12.2-4.el8.noarch 2.2 MB/s | 457 kB 00:00 2023-12-11T19:20:22.747 INFO:teuthology.orchestra.run.smithi167.stdout:(115/132): python3-numpy-f2py-1.14.3-10.el8.x86 3.8 MB/s | 225 kB 00:00 2023-12-11T19:20:22.770 INFO:teuthology.orchestra.run.smithi066.stdout:(31/132): ceph-mgr-modules-core-19.0.0-119.ga62 3.2 MB/s | 243 kB 00:00 2023-12-11T19:20:22.780 INFO:teuthology.orchestra.run.smithi070.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 439 kB/s | 59 kB 00:00 2023-12-11T19:20:22.797 INFO:teuthology.orchestra.run.smithi070.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2023-12-11T19:20:22.806 INFO:teuthology.orchestra.run.smithi167.stdout:(116/132): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-12-11T19:20:22.820 INFO:teuthology.orchestra.run.smithi066.stdout:(32/132): ceph-mgr-rook-19.0.0-119.ga62fda73.el 1.0 MB/s | 50 kB 00:00 2023-12-11T19:20:22.831 INFO:teuthology.orchestra.run.smithi167.stdout:(117/132): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.5 MB/s | 126 kB 00:00 2023-12-11T19:20:22.839 INFO:teuthology.orchestra.run.smithi070.stdout:(56/132): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2023-12-11T19:20:22.856 INFO:teuthology.orchestra.run.smithi167.stdout:(118/132): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-12-11T19:20:22.871 INFO:teuthology.orchestra.run.smithi066.stdout:(33/132): ceph-prometheus-alerts-19.0.0-119.ga6 300 kB/s | 15 kB 00:00 2023-12-11T19:20:22.882 INFO:teuthology.orchestra.run.smithi167.stdout:(119/132): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-12-11T19:20:22.897 INFO:teuthology.orchestra.run.smithi070.stdout:(57/132): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-12-11T19:20:22.940 INFO:teuthology.orchestra.run.smithi070.stdout:(58/132): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-12-11T19:20:22.946 INFO:teuthology.orchestra.run.smithi066.stdout:(34/132): ceph-volume-19.0.0-119.ga62fda73.el8. 3.4 MB/s | 262 kB 00:00 2023-12-11T19:20:22.949 INFO:teuthology.orchestra.run.smithi167.stdout:(120/132): python3-numpy-1.14.3-10.el8.x86_64.r 6.8 MB/s | 3.7 MB 00:00 2023-12-11T19:20:22.991 INFO:teuthology.orchestra.run.smithi167.stdout:(121/132): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-12-11T19:20:23.016 INFO:teuthology.orchestra.run.smithi167.stdout:(122/132): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-12-11T19:20:23.023 INFO:teuthology.orchestra.run.smithi070.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-12-11T19:20:23.038 INFO:teuthology.orchestra.run.smithi066.stdout:(35/132): cephadm-19.0.0-119.ga62fda73.el8.noar 4.4 MB/s | 415 kB 00:00 2023-12-11T19:20:23.057 INFO:teuthology.orchestra.run.smithi070.stdout:(60/132): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-12-11T19:20:23.058 INFO:teuthology.orchestra.run.smithi167.stdout:(123/132): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-12-11T19:20:23.082 INFO:teuthology.orchestra.run.smithi070.stdout:(61/132): python3-rsa-4.9-2.el8.noarch.rpm 452 kB/s | 64 kB 00:00 2023-12-11T19:20:23.088 INFO:teuthology.orchestra.run.smithi066.stdout:(36/132): cephfs-top-19.0.0-119.ga62fda73.el8.n 364 kB/s | 18 kB 00:00 2023-12-11T19:20:23.100 INFO:teuthology.orchestra.run.smithi167.stdout:(124/132): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-12-11T19:20:23.132 INFO:teuthology.orchestra.run.smithi070.stdout:(62/132): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-12-11T19:20:23.157 INFO:teuthology.orchestra.run.smithi070.stdout:(63/132): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-12-11T19:20:23.183 INFO:teuthology.orchestra.run.smithi070.stdout:(64/132): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-12-11T19:20:23.225 INFO:teuthology.orchestra.run.smithi167.stdout:(125/132): openblas-threads-0.3.15-3.el8.x86_64 14 MB/s | 4.9 MB 00:00 2023-12-11T19:20:23.233 INFO:teuthology.orchestra.run.smithi070.stdout:(65/132): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-12-11T19:20:23.292 INFO:teuthology.orchestra.run.smithi070.stdout:(66/132): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-12-11T19:20:23.317 INFO:teuthology.orchestra.run.smithi070.stdout:(67/132): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-12-11T19:20:23.325 INFO:teuthology.orchestra.run.smithi167.stdout:(126/132): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-12-11T19:20:23.342 INFO:teuthology.orchestra.run.smithi070.stdout:(68/132): python3-websocket-client-0.56.0-5.el8 563 kB/s | 61 kB 00:00 2023-12-11T19:20:23.367 INFO:teuthology.orchestra.run.smithi167.stdout:(127/132): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-12-11T19:20:23.392 INFO:teuthology.orchestra.run.smithi070.stdout:(69/132): python3-webtest-2.0.33-1.el8.noarch.r 850 kB/s | 85 kB 00:00 2023-12-11T19:20:23.426 INFO:teuthology.orchestra.run.smithi070.stdout:(70/132): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2023-12-11T19:20:23.434 INFO:teuthology.orchestra.run.smithi167.stdout:(128/132): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-12-11T19:20:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:(71/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-12-11T19:20:23.455 INFO:teuthology.orchestra.run.smithi066.stdout:(37/132): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2023-12-11T19:20:23.501 INFO:teuthology.orchestra.run.smithi070.stdout:(72/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 625 kB/s | 47 kB 00:00 2023-12-11T19:20:23.509 INFO:teuthology.orchestra.run.smithi167.stdout:(129/132): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-12-11T19:20:23.535 INFO:teuthology.orchestra.run.smithi070.stdout:(73/132): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-12-11T19:20:23.539 INFO:teuthology.orchestra.run.smithi066.stdout:(38/132): gperftools-libs-2.7-9.el8.x86_64.rpm 680 kB/s | 306 kB 00:00 2023-12-11T19:20:23.561 INFO:teuthology.orchestra.run.smithi070.stdout:(74/132): python3-jwt-1.6.1-2.el8.noarch.rpm 725 kB/s | 43 kB 00:00 2023-12-11T19:20:23.568 INFO:teuthology.orchestra.run.smithi167.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-12-11T19:20:23.594 INFO:teuthology.orchestra.run.smithi070.stdout:(75/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-12-11T19:20:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:(76/132): keyutils-libs-devel-1.5.10-9.el8.x86_ 822 kB/s | 48 kB 00:00 2023-12-11T19:20:23.648 INFO:teuthology.orchestra.run.smithi066.stdout:(39/132): libunwind-1.3.1-3.el8.x86_64.rpm 693 kB/s | 75 kB 00:00 2023-12-11T19:20:23.662 INFO:teuthology.orchestra.run.smithi070.stdout:(77/132): thrift-0.13.0-2.el8.x86_64.rpm 5.4 MB/s | 1.7 MB 00:00 2023-12-11T19:20:23.687 INFO:teuthology.orchestra.run.smithi070.stdout:(78/132): libsepol-devel-2.9-3.el8.x86_64.rpm 946 kB/s | 87 kB 00:00 2023-12-11T19:20:23.721 INFO:teuthology.orchestra.run.smithi070.stdout:(79/132): python3-requests-oauthlib-1.0.0-1.el8 426 kB/s | 43 kB 00:00 2023-12-11T19:20:23.771 INFO:teuthology.orchestra.run.smithi070.stdout:(80/132): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-12-11T19:20:23.796 INFO:teuthology.orchestra.run.smithi070.stdout:(81/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.2 MB/s | 229 kB 00:00 2023-12-11T19:20:23.821 INFO:teuthology.orchestra.run.smithi070.stdout:(82/132): libaio-devel-0.3.112-1.el8.x86_64.rpm 374 kB/s | 19 kB 00:00 2023-12-11T19:20:23.847 INFO:teuthology.orchestra.run.smithi070.stdout:(83/132): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-12-11T19:20:23.864 INFO:teuthology.orchestra.run.smithi066.stdout:(40/132): liboath-2.6.2-3.el8.x86_64.rpm 144 kB/s | 59 kB 00:00 2023-12-11T19:20:23.889 INFO:teuthology.orchestra.run.smithi070.stdout:(84/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-12-11T19:20:23.890 INFO:teuthology.orchestra.run.smithi066.stdout:(41/132): luarocks-3.5.0-1.el8.noarch.rpm 691 kB/s | 167 kB 00:00 2023-12-11T19:20:23.914 INFO:teuthology.orchestra.run.smithi070.stdout:(85/132): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-12-11T19:20:23.956 INFO:teuthology.orchestra.run.smithi070.stdout:(86/132): python3-cryptography-3.2.1-5.el8.x86_ 4.1 MB/s | 559 kB 00:00 2023-12-11T19:20:23.965 INFO:teuthology.orchestra.run.smithi066.stdout:(42/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 582 kB/s | 44 kB 00:00 2023-12-11T19:20:23.968 INFO:teuthology.orchestra.run.smithi167.stdout:(131/132): librados2-19.0.0-119.ga62fda73.el8.x 6.3 MB/s | 3.8 MB 00:00 2023-12-11T19:20:23.989 INFO:teuthology.orchestra.run.smithi070.stdout:(87/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-12-11T19:20:24.023 INFO:teuthology.orchestra.run.smithi070.stdout:(88/132): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-12-11T19:20:24.027 INFO:teuthology.orchestra.run.smithi167.stdout:(132/132): librbd1-19.0.0-119.ga62fda73.el8.x86 6.3 MB/s | 3.7 MB 00:00 2023-12-11T19:20:24.031 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:20:24.031 INFO:teuthology.orchestra.run.smithi167.stdout:Total 13 MB/s | 209 MB 00:15 2023-12-11T19:20:24.048 INFO:teuthology.orchestra.run.smithi070.stdout:(89/132): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-12-11T19:20:24.073 INFO:teuthology.orchestra.run.smithi066.stdout:(43/132): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-12-11T19:20:24.074 INFO:teuthology.orchestra.run.smithi070.stdout:(90/132): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-11T19:20:24.091 INFO:teuthology.orchestra.run.smithi070.stdout:(91/132): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-12-11T19:20:24.115 INFO:teuthology.orchestra.run.smithi066.stdout:(44/132): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-12-11T19:20:24.116 INFO:teuthology.orchestra.run.smithi070.stdout:(92/132): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-12-11T19:20:24.133 INFO:teuthology.orchestra.run.smithi070.stdout:(93/132): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-12-11T19:20:24.149 INFO:teuthology.orchestra.run.smithi066.stdout:(45/132): python3-cachetools-3.1.1-4.el8.noarch 439 kB/s | 33 kB 00:00 2023-12-11T19:20:24.150 INFO:teuthology.orchestra.run.smithi070.stdout:(94/132): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-12-11T19:20:24.191 INFO:teuthology.orchestra.run.smithi066.stdout:(46/132): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-12-11T19:20:24.200 INFO:teuthology.orchestra.run.smithi070.stdout:(95/132): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-12-11T19:20:24.225 INFO:teuthology.orchestra.run.smithi070.stdout:(96/132): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-12-11T19:20:24.249 INFO:teuthology.orchestra.run.smithi066.stdout:(47/132): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-12-11T19:20:24.251 INFO:teuthology.orchestra.run.smithi070.stdout:(97/132): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-12-11T19:20:24.293 INFO:teuthology.orchestra.run.smithi070.stdout:(98/132): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-12-11T19:20:24.300 INFO:teuthology.orchestra.run.smithi066.stdout:(48/132): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-12-11T19:20:24.334 INFO:teuthology.orchestra.run.smithi066.stdout:(49/132): python3-google-auth-1.1.1-10.el8.noar 969 kB/s | 81 kB 00:00 2023-12-11T19:20:24.343 INFO:teuthology.orchestra.run.smithi070.stdout:(99/132): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.0 MB/s | 2.3 MB 00:00 2023-12-11T19:20:24.376 INFO:teuthology.orchestra.run.smithi066.stdout:(50/132): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-12-11T19:20:24.398 INFO:teuthology.orchestra.run.smithi070.stdout:(100/132): python3-pytz-2017.2-9.el8.noarch.rpm 518 kB/s | 54 kB 00:00 2023-12-11T19:20:24.409 INFO:teuthology.orchestra.run.smithi066.stdout:(51/132): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-12-11T19:20:24.435 INFO:teuthology.orchestra.run.smithi070.stdout:(101/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2023-12-11T19:20:24.479 INFO:teuthology.orchestra.run.smithi070.stdout:(102/132): socat-1.7.4.1-1.el8.x86_64.rpm 3.9 MB/s | 323 kB 00:00 2023-12-11T19:20:24.501 INFO:teuthology.orchestra.run.smithi066.stdout:(52/132): python3-logutils-0.3.5-11.el8.noarch. 533 kB/s | 49 kB 00:00 2023-12-11T19:20:24.505 INFO:teuthology.orchestra.run.smithi070.stdout:(103/132): python3-markupsafe-0.23-19.el8.x86_6 564 kB/s | 39 kB 00:00 2023-12-11T19:20:24.552 INFO:teuthology.orchestra.run.smithi066.stdout:(53/132): python3-kubernetes-11.0.0-6.el8.noarc 6.7 MB/s | 1.2 MB 00:00 2023-12-11T19:20:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:(104/132): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-12-11T19:20:24.585 INFO:teuthology.orchestra.run.smithi066.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 705 kB/s | 59 kB 00:00 2023-12-11T19:20:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:(105/132): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-12-11T19:20:24.614 INFO:teuthology.orchestra.run.smithi070.stdout:(106/132): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-12-11T19:20:24.627 INFO:teuthology.orchestra.run.smithi066.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 802 kB/s | 60 kB 00:00 2023-12-11T19:20:24.647 INFO:teuthology.orchestra.run.smithi070.stdout:(107/132): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-12-11T19:20:24.694 INFO:teuthology.orchestra.run.smithi066.stdout:(56/132): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-12-11T19:20:24.698 INFO:teuthology.orchestra.run.smithi070.stdout:(108/132): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-12-11T19:20:24.728 INFO:teuthology.orchestra.run.smithi066.stdout:(57/132): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-12-11T19:20:24.731 INFO:teuthology.orchestra.run.smithi070.stdout:(109/132): python3-jsonpatch-1.21-2.el8.noarch. 805 kB/s | 27 kB 00:00 2023-12-11T19:20:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:(110/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-11T19:20:24.786 INFO:teuthology.orchestra.run.smithi066.stdout:(58/132): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2023-12-11T19:20:24.807 INFO:teuthology.orchestra.run.smithi070.stdout:(111/132): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-12-11T19:20:24.812 INFO:teuthology.orchestra.run.smithi066.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-12-11T19:20:24.862 INFO:teuthology.orchestra.run.smithi066.stdout:(60/132): python3-rsa-4.9-2.el8.noarch.rpm 858 kB/s | 64 kB 00:00 2023-12-11T19:20:24.887 INFO:teuthology.orchestra.run.smithi066.stdout:(61/132): python3-simplegeneric-0.8.1-17.el8.no 250 kB/s | 19 kB 00:00 2023-12-11T19:20:24.932 INFO:teuthology.orchestra.run.smithi070.stdout:(112/132): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-12-11T19:20:24.938 INFO:teuthology.orchestra.run.smithi066.stdout:(62/132): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-12-11T19:20:24.971 INFO:teuthology.orchestra.run.smithi066.stdout:(63/132): python3-tempora-1.14.1-5.el8.noarch.r 350 kB/s | 29 kB 00:00 2023-12-11T19:20:24.982 INFO:teuthology.orchestra.run.smithi070.stdout:(113/132): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-12-11T19:20:25.013 INFO:teuthology.orchestra.run.smithi066.stdout:(64/132): python3-trustme-0.6.0-4.el8.noarch.rp 367 kB/s | 27 kB 00:00 2023-12-11T19:20:25.055 INFO:teuthology.orchestra.run.smithi066.stdout:(65/132): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-12-11T19:20:25.074 INFO:teuthology.orchestra.run.smithi070.stdout:(114/132): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-12-11T19:20:25.109 INFO:teuthology.orchestra.run.smithi070.stdout:(115/132): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2023-12-11T19:20:25.114 INFO:teuthology.orchestra.run.smithi066.stdout:(66/132): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-12-11T19:20:25.133 INFO:teuthology.orchestra.run.smithi070.stdout:(116/132): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-12-11T19:20:25.139 INFO:teuthology.orchestra.run.smithi066.stdout:(67/132): python3-websocket-client-0.56.0-5.el8 734 kB/s | 61 kB 00:00 2023-12-11T19:20:25.158 INFO:teuthology.orchestra.run.smithi070.stdout:(117/132): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-12-11T19:20:25.181 INFO:teuthology.orchestra.run.smithi066.stdout:(68/132): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2023-12-11T19:20:25.184 INFO:teuthology.orchestra.run.smithi070.stdout:(118/132): python3-jsonpointer-1.10-11.el8.noar 396 kB/s | 20 kB 00:00 2023-12-11T19:20:25.209 INFO:teuthology.orchestra.run.smithi070.stdout:(119/132): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-12-11T19:20:25.214 INFO:teuthology.orchestra.run.smithi066.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-12-11T19:20:25.240 INFO:teuthology.orchestra.run.smithi066.stdout:(70/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-12-11T19:20:25.251 INFO:teuthology.orchestra.run.smithi070.stdout:(120/132): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-12-11T19:20:25.293 INFO:teuthology.orchestra.run.smithi070.stdout:(121/132): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-12-11T19:20:25.460 INFO:teuthology.orchestra.run.smithi070.stdout:(122/132): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-12-11T19:20:25.465 INFO:teuthology.orchestra.run.smithi066.stdout:(71/132): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-12-11T19:20:25.502 INFO:teuthology.orchestra.run.smithi070.stdout:(123/132): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-12-11T19:20:25.523 INFO:teuthology.orchestra.run.smithi066.stdout:(72/132): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-12-11T19:20:25.544 INFO:teuthology.orchestra.run.smithi070.stdout:(124/132): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-12-11T19:20:25.557 INFO:teuthology.orchestra.run.smithi066.stdout:(73/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-11T19:20:25.599 INFO:teuthology.orchestra.run.smithi066.stdout:(74/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-12-11T19:20:25.628 INFO:teuthology.orchestra.run.smithi070.stdout:(125/132): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-12-11T19:20:25.649 INFO:teuthology.orchestra.run.smithi066.stdout:(75/132): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-12-11T19:20:25.674 INFO:teuthology.orchestra.run.smithi066.stdout:(76/132): python3-markupsafe-0.23-19.el8.x86_64 1.5 MB/s | 39 kB 00:00 2023-12-11T19:20:25.701 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-12-11T19:20:25.750 INFO:teuthology.orchestra.run.smithi066.stdout:(77/132): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-12-11T19:20:25.778 INFO:teuthology.orchestra.run.smithi070.stdout:(126/132): openblas-0.3.15-3.el8.x86_64.rpm 9.8 MB/s | 4.7 MB 00:00 2023-12-11T19:20:25.800 INFO:teuthology.orchestra.run.smithi066.stdout:(78/132): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-12-11T19:20:25.811 INFO:teuthology.orchestra.run.smithi070.stdout:(127/132): postgresql-contrib-10.21-2.module+el 4.3 MB/s | 808 kB 00:00 2023-12-11T19:20:25.841 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-12-11T19:20:25.841 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-12-11T19:20:25.886 INFO:teuthology.orchestra.run.smithi070.stdout:(128/132): postgresql-10.21-2.module+el8.6.0+15 20 MB/s | 1.5 MB 00:00 2023-12-11T19:20:25.959 INFO:teuthology.orchestra.run.smithi066.stdout:(79/132): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-12-11T19:20:25.978 INFO:teuthology.orchestra.run.smithi070.stdout:(129/132): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-12-11T19:20:25.992 INFO:teuthology.orchestra.run.smithi066.stdout:(80/132): python3-mako-1.0.6-13.el8.noarch.rpm 818 kB/s | 157 kB 00:00 2023-12-11T19:20:26.026 INFO:teuthology.orchestra.run.smithi066.stdout:(81/132): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-12-11T19:20:26.037 INFO:teuthology.orchestra.run.smithi070.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-12-11T19:20:26.077 INFO:teuthology.orchestra.run.smithi066.stdout:(82/132): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-12-11T19:20:26.110 INFO:teuthology.orchestra.run.smithi066.stdout:(83/132): python3-jsonpatch-1.21-2.el8.noarch.r 806 kB/s | 27 kB 00:00 2023-12-11T19:20:26.152 INFO:teuthology.orchestra.run.smithi066.stdout:(84/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-11T19:20:26.186 INFO:teuthology.orchestra.run.smithi066.stdout:(85/132): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-12-11T19:20:26.319 INFO:teuthology.orchestra.run.smithi066.stdout:(86/132): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-12-11T19:20:26.363 INFO:teuthology.orchestra.run.smithi070.stdout:(131/132): librados2-19.0.0-119.ga62fda73.el8.x 6.8 MB/s | 3.8 MB 00:00 2023-12-11T19:20:26.370 INFO:teuthology.orchestra.run.smithi066.stdout:(87/132): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-12-11T19:20:26.412 INFO:teuthology.orchestra.run.smithi066.stdout:(88/132): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-12-11T19:20:26.470 INFO:teuthology.orchestra.run.smithi066.stdout:(89/132): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2023-12-11T19:20:26.495 INFO:teuthology.orchestra.run.smithi066.stdout:(90/132): uuid-1.6.2-43.el8.x86_64.rpm 768 kB/s | 64 kB 00:00 2023-12-11T19:20:26.521 INFO:teuthology.orchestra.run.smithi066.stdout:(91/132): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-12-11T19:20:26.546 INFO:teuthology.orchestra.run.smithi066.stdout:(92/132): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-12-11T19:20:26.563 INFO:teuthology.orchestra.run.smithi066.stdout:(93/132): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-12-11T19:20:26.604 INFO:teuthology.orchestra.run.smithi070.stdout:(132/132): librbd1-19.0.0-119.ga62fda73.el8.x86 5.2 MB/s | 3.7 MB 00:00 2023-12-11T19:20:26.605 INFO:teuthology.orchestra.run.smithi066.stdout:(94/132): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-12-11T19:20:26.608 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:20:26.609 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 209 MB 00:16 2023-12-11T19:20:26.647 INFO:teuthology.orchestra.run.smithi066.stdout:(95/132): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-12-11T19:20:26.789 INFO:teuthology.orchestra.run.smithi066.stdout:(96/132): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-12-11T19:20:26.831 INFO:teuthology.orchestra.run.smithi066.stdout:(97/132): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-12-11T19:20:26.914 INFO:teuthology.orchestra.run.smithi066.stdout:(98/132): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-12-11T19:20:26.948 INFO:teuthology.orchestra.run.smithi066.stdout:(99/132): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-12-11T19:20:27.007 INFO:teuthology.orchestra.run.smithi066.stdout:(100/132): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-12-11T19:20:27.090 INFO:teuthology.orchestra.run.smithi066.stdout:(101/132): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-12-11T19:20:27.132 INFO:teuthology.orchestra.run.smithi066.stdout:(102/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-12-11T19:20:27.207 INFO:teuthology.orchestra.run.smithi066.stdout:(103/132): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-12-11T19:20:27.241 INFO:teuthology.orchestra.run.smithi066.stdout:(104/132): libselinux-devel-2.9-5.el8.x86_64.rp 1.8 MB/s | 200 kB 00:00 2023-12-11T19:20:27.266 INFO:teuthology.orchestra.run.smithi066.stdout:(105/132): python3-jwt-1.6.1-2.el8.noarch.rpm 730 kB/s | 43 kB 00:00 2023-12-11T19:20:27.424 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-12-11T19:20:27.429 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-12-11T19:20:27.726 INFO:teuthology.orchestra.run.smithi066.stdout:(106/132): ceph-test-19.0.0-119.ga62fda73.el8.x 6.6 MB/s | 49 MB 00:07 2023-12-11T19:20:27.751 INFO:teuthology.orchestra.run.smithi066.stdout:(107/132): keyutils-libs-devel-1.5.10-9.el8.x86 99 kB/s | 48 kB 00:00 2023-12-11T19:20:27.784 INFO:teuthology.orchestra.run.smithi066.stdout:(108/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 406 kB/s | 220 kB 00:00 2023-12-11T19:20:27.801 INFO:teuthology.orchestra.run.smithi066.stdout:(109/132): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-12-11T19:20:27.826 INFO:teuthology.orchestra.run.smithi066.stdout:(110/132): libsepol-devel-2.9-3.el8.x86_64.rpm 885 kB/s | 87 kB 00:00 2023-12-11T19:20:27.852 INFO:teuthology.orchestra.run.smithi066.stdout:(111/132): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2023-12-11T19:20:27.885 INFO:teuthology.orchestra.run.smithi066.stdout:(112/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-12-11T19:20:27.935 INFO:teuthology.orchestra.run.smithi066.stdout:(113/132): libaio-devel-0.3.112-1.el8.x86_64.rp 225 kB/s | 19 kB 00:00 2023-12-11T19:20:27.961 INFO:teuthology.orchestra.run.smithi066.stdout:(114/132): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-12-11T19:20:27.994 INFO:teuthology.orchestra.run.smithi066.stdout:(115/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-12-11T19:20:28.020 INFO:teuthology.orchestra.run.smithi066.stdout:(116/132): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-12-11T19:20:28.053 INFO:teuthology.orchestra.run.smithi066.stdout:(117/132): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-12-11T19:20:28.087 INFO:teuthology.orchestra.run.smithi066.stdout:(118/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-12-11T19:20:28.112 INFO:teuthology.orchestra.run.smithi066.stdout:(119/132): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-12-11T19:20:28.137 INFO:teuthology.orchestra.run.smithi066.stdout:(120/132): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-12-11T19:20:28.163 INFO:teuthology.orchestra.run.smithi066.stdout:(121/132): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-11T19:20:28.188 INFO:teuthology.orchestra.run.smithi066.stdout:(122/132): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-12-11T19:20:28.213 INFO:teuthology.orchestra.run.smithi066.stdout:(123/132): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-12-11T19:20:28.238 INFO:teuthology.orchestra.run.smithi066.stdout:(124/132): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-12-11T19:20:28.264 INFO:teuthology.orchestra.run.smithi066.stdout:(125/132): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-12-11T19:20:28.265 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-12-11T19:20:28.306 INFO:teuthology.orchestra.run.smithi066.stdout:(126/132): libquadmath-8.5.0-10.1.el8_6.x86_64. 4.0 MB/s | 171 kB 00:00 2023-12-11T19:20:28.331 INFO:teuthology.orchestra.run.smithi066.stdout:(127/132): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.8 MB/s | 644 kB 00:00 2023-12-11T19:20:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-12-11T19:20:28.403 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-12-11T19:20:28.506 INFO:teuthology.orchestra.run.smithi066.stdout:(128/132): NetworkManager-initscripts-updown-1. 687 kB/s | 138 kB 00:00 2023-12-11T19:20:28.656 INFO:teuthology.orchestra.run.smithi066.stdout:(129/132): openssl-devel-1.1.1k-7.el8_6.x86_64. 3.9 MB/s | 2.3 MB 00:00 2023-12-11T19:20:28.723 INFO:teuthology.orchestra.run.smithi066.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-12-11T19:20:28.890 INFO:teuthology.orchestra.run.smithi066.stdout:(131/132): librados2-19.0.0-119.ga62fda73.el8.x 6.7 MB/s | 3.8 MB 00:00 2023-12-11T19:20:29.115 INFO:teuthology.orchestra.run.smithi066.stdout:(132/132): librbd1-19.0.0-119.ga62fda73.el8.x86 6.1 MB/s | 3.7 MB 00:00 2023-12-11T19:20:29.119 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:20:29.119 INFO:teuthology.orchestra.run.smithi066.stdout:Total 11 MB/s | 209 MB 00:19 2023-12-11T19:20:29.990 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-12-11T19:20:29.995 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-12-11T19:20:30.844 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2023-12-11T19:20:30.997 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2023-12-11T19:20:30.998 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2023-12-11T19:20:32.496 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-12-11T19:20:32.627 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2023-12-11T19:20:32.632 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2023-12-11T19:20:32.678 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-11T19:20:32.914 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-11T19:20:33.105 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T19:20:33.246 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T19:20:33.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T19:20:33.973 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T19:20:34.171 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T19:20:34.678 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T19:20:34.896 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:20:35.057 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:20:35.209 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-12-11T19:20:35.296 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:20:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-11T19:20:35.414 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:20:35.602 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:20:35.618 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-11T19:20:35.793 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T19:20:36.137 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T19:20:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T19:20:36.766 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/135 2023-12-11T19:20:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T19:20:36.973 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:20:37.067 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T19:20:37.082 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:20:37.382 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-11T19:20:37.539 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T19:20:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2023-12-11T19:20:37.609 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-11T19:20:37.709 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:20:37.757 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-11T19:20:37.866 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:20:37.917 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-11T19:20:38.051 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:20:38.075 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-11T19:20:38.111 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/135 2023-12-11T19:20:38.173 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:20:38.283 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/135 2023-12-11T19:20:38.349 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:20:38.363 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-11T19:20:38.439 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/135 2023-12-11T19:20:38.529 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:20:38.642 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:20:38.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T19:20:38.900 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/135 2023-12-11T19:20:39.062 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T19:20:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/135 2023-12-11T19:20:39.344 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/135 2023-12-11T19:20:39.358 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:20:39.404 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/135 2023-12-11T19:20:39.465 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:20:39.538 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/135 2023-12-11T19:20:39.616 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:20:39.723 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-11T19:20:39.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/135 2023-12-11T19:20:39.936 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T19:20:39.964 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-11T19:20:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T19:20:40.113 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:20:40.291 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:20:40.304 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T19:20:40.386 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-11T19:20:40.451 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:20:40.609 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-11T19:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:20:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:20:40.764 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:20:40.891 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:20:40.978 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T19:20:41.133 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T19:20:41.319 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T19:20:41.387 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T19:20:41.556 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/135 2023-12-11T19:20:41.620 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T19:20:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/135 2023-12-11T19:20:41.774 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:20:41.871 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:20:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/135 2023-12-11T19:20:42.044 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T19:20:42.116 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T19:20:42.126 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-11T19:20:42.236 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T19:20:42.287 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T19:20:42.355 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-11T19:20:42.475 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T19:20:42.769 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-11T19:20:43.008 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-11T19:20:43.109 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T19:20:43.157 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T19:20:43.174 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:20:43.308 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:20:43.325 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T19:20:43.419 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/135 2023-12-11T19:20:43.474 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T19:20:43.570 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T19:20:43.622 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/135 2023-12-11T19:20:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 17/135 2023-12-11T19:20:43.869 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 17/135 2023-12-11T19:20:43.924 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-11T19:20:43.990 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T19:20:44.032 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 18/135 2023-12-11T19:20:44.102 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 27/135 2023-12-11T19:20:44.174 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T19:20:44.249 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 28/135 2023-12-11T19:20:44.531 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 29/135 2023-12-11T19:20:44.728 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T19:20:44.748 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 18/135 2023-12-11T19:20:44.933 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 19/135 2023-12-11T19:20:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T19:20:45.290 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/135 2023-12-11T19:20:45.508 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/135 2023-12-11T19:20:45.645 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 19/135 2023-12-11T19:20:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-11T19:20:45.823 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.15-3.el8.x86_64 20/135 2023-12-11T19:20:45.923 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 27/135 2023-12-11T19:20:46.062 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 28/135 2023-12-11T19:20:46.575 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 29/135 2023-12-11T19:20:46.705 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 20/135 2023-12-11T19:20:46.802 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T19:20:46.824 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T19:20:46.948 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 21/135 2023-12-11T19:20:47.188 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 22/135 2023-12-11T19:20:47.406 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/135 2023-12-11T19:20:47.625 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/135 2023-12-11T19:20:47.872 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 25/135 2023-12-11T19:20:47.999 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/135 2023-12-11T19:20:48.140 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-11T19:20:48.315 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 27/135 2023-12-11T19:20:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/135 2023-12-11T19:20:48.454 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 28/135 2023-12-11T19:20:48.628 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/135 2023-12-11T19:20:48.817 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T19:20:48.911 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/135 2023-12-11T19:20:49.131 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-11T19:20:49.349 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-11T19:20:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 29/135 2023-12-11T19:20:49.576 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 37/135 2023-12-11T19:20:49.751 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 38/135 2023-12-11T19:20:49.886 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 30/135 2023-12-11T19:20:49.939 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/135 2023-12-11T19:20:49.985 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 39/135 2023-12-11T19:20:50.153 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/135 2023-12-11T19:20:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 31/135 2023-12-11T19:20:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/135 2023-12-11T19:20:50.384 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/135 2023-12-11T19:20:50.455 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/135 2023-12-11T19:20:50.560 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T19:20:50.604 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/135 2023-12-11T19:20:50.648 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/135 2023-12-11T19:20:50.657 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T19:20:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/135 2023-12-11T19:20:50.952 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/135 2023-12-11T19:20:51.093 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-11T19:20:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.1-7.el8.noarch 44/135 2023-12-11T19:20:51.319 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-11T19:20:51.427 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T19:20:51.527 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 37/135 2023-12-11T19:20:51.528 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T19:20:51.705 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 38/135 2023-12-11T19:20:51.763 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.6-3.el8.x86_64 46/135 2023-12-11T19:20:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 39/135 2023-12-11T19:20:51.965 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/135 2023-12-11T19:20:52.091 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/135 2023-12-11T19:20:52.127 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/135 2023-12-11T19:20:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/135 2023-12-11T19:20:52.363 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/135 2023-12-11T19:20:52.431 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T19:20:52.521 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T19:20:52.572 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/135 2023-12-11T19:20:52.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/135 2023-12-11T19:20:52.808 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/135 2023-12-11T19:20:52.844 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/135 2023-12-11T19:20:53.028 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.1-7.el8.noarch 44/135 2023-12-11T19:20:53.059 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/135 2023-12-11T19:20:53.158 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/135 2023-12-11T19:20:53.197 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T19:20:53.298 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T19:20:53.309 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/135 2023-12-11T19:20:53.354 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-11T19:20:53.519 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-11T19:20:53.529 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-3.el8.x86_64 46/135 2023-12-11T19:20:53.710 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/135 2023-12-11T19:20:53.721 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 37/135 2023-12-11T19:20:53.788 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/135 2023-12-11T19:20:53.880 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 38/135 2023-12-11T19:20:53.914 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/135 2023-12-11T19:20:54.094 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 39/135 2023-12-11T19:20:54.100 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/135 2023-12-11T19:20:54.117 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/135 2023-12-11T19:20:54.266 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 40/135 2023-12-11T19:20:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/135 2023-12-11T19:20:54.417 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/135 2023-12-11T19:20:54.445 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/135 2023-12-11T19:20:54.451 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 41/135 2023-12-11T19:20:54.659 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.7-15.el8.x86_64 42/135 2023-12-11T19:20:54.674 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/135 2023-12-11T19:20:54.695 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/135 2023-12-11T19:20:54.813 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 43/135 2023-12-11T19:20:54.919 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/135 2023-12-11T19:20:54.929 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/135 2023-12-11T19:20:55.056 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 44/135 2023-12-11T19:20:55.254 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/135 2023-12-11T19:20:55.273 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 45/135 2023-12-11T19:20:55.379 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/135 2023-12-11T19:20:55.537 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/135 2023-12-11T19:20:55.653 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/135 2023-12-11T19:20:55.706 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 46/135 2023-12-11T19:20:55.730 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/135 2023-12-11T19:20:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/135 2023-12-11T19:20:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 47/135 2023-12-11T19:20:56.040 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/135 2023-12-11T19:20:56.181 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/135 2023-12-11T19:20:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/135 2023-12-11T19:20:56.261 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/135 2023-12-11T19:20:56.465 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/135 2023-12-11T19:20:56.473 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 49/135 2023-12-11T19:20:56.508 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/135 2023-12-11T19:20:56.693 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 50/135 2023-12-11T19:20:56.812 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/135 2023-12-11T19:20:56.894 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/135 2023-12-11T19:20:57.014 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 51/135 2023-12-11T19:20:57.106 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/135 2023-12-11T19:20:57.194 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/135 2023-12-11T19:20:57.249 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 52/135 2023-12-11T19:20:57.295 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/135 2023-12-11T19:20:57.418 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 53/135 2023-12-11T19:20:57.493 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/135 2023-12-11T19:20:57.524 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/135 2023-12-11T19:20:57.671 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 54/135 2023-12-11T19:20:57.729 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/135 2023-12-11T19:20:57.737 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/135 2023-12-11T19:20:57.849 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 55/135 2023-12-11T19:20:57.961 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-11T19:20:57.968 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/135 2023-12-11T19:20:58.018 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 56/135 2023-12-11T19:20:58.135 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T19:20:58.321 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/135 2023-12-11T19:20:58.389 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 57/135 2023-12-11T19:20:58.420 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-11T19:20:58.607 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/135 2023-12-11T19:20:58.643 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-11T19:20:58.677 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 58/135 2023-12-11T19:20:58.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-11T19:20:58.869 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/135 2023-12-11T19:20:58.932 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 59/135 2023-12-11T19:20:58.965 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-11T19:20:59.139 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 60/135 2023-12-11T19:20:59.149 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/135 2023-12-11T19:20:59.166 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-11T19:20:59.331 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-11T19:20:59.340 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 61/135 2023-12-11T19:20:59.368 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-11T19:20:59.493 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 62/135 2023-12-11T19:20:59.528 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T19:20:59.593 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-11T19:20:59.697 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-43.el8.x86_64 63/135 2023-12-11T19:20:59.780 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-11T19:20:59.795 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 63/135 2023-12-11T19:20:59.845 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-11T19:21:00.009 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-11T19:21:00.058 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 64/135 2023-12-11T19:21:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-11T19:21:00.236 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-11T19:21:00.267 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.1-7.el8.noarch 65/135 2023-12-11T19:21:00.275 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-11T19:21:00.430 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 66/135 2023-12-11T19:21:00.450 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-11T19:21:00.526 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-11T19:21:00.533 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 66/135 2023-12-11T19:21:00.675 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-11T19:21:00.734 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-11T19:21:00.764 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : jq-1.6-3.el8.x86_64 67/135 2023-12-11T19:21:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-11T19:21:00.933 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 68/135 2023-12-11T19:21:01.013 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-11T19:21:01.099 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-11T19:21:01.108 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-11T19:21:01.235 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-11T19:21:01.274 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T19:21:01.299 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-11T19:21:01.432 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-11T19:21:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-11T19:21:01.535 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-11T19:21:01.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-11T19:21:01.747 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-11T19:21:01.755 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-11T19:21:01.902 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-11T19:21:01.975 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-11T19:21:02.060 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-11T19:21:02.193 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-11T19:21:02.263 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-11T19:21:02.452 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-11T19:21:02.472 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-11T19:21:02.645 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-11T19:21:02.671 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-11T19:21:02.834 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-11T19:21:02.896 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-11T19:21:03.004 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-11T19:21:03.065 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-11T19:21:03.142 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-11T19:21:03.222 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-11T19:21:03.362 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-11T19:21:03.411 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-11T19:21:03.612 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-11T19:21:03.665 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-11T19:21:03.874 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-11T19:21:04.043 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-11T19:21:04.073 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-11T19:21:04.266 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-11T19:21:04.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-11T19:21:04.446 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-11T19:21:04.515 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-11T19:21:04.664 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-11T19:21:04.764 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-11T19:21:04.892 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-11T19:21:05.033 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-11T19:21:05.125 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-11T19:21:05.408 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-11T19:21:05.760 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-11T19:21:05.917 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-11T19:21:06.022 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-11T19:21:06.210 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-11T19:21:06.231 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-11T19:21:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-11T19:21:06.470 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-11T19:21:06.838 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-11T19:21:07.200 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-11T19:21:07.438 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-11T19:21:07.629 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-11T19:21:07.805 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-11T19:21:10.530 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:10.817 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:11.335 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:11.493 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:11.560 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:11.834 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T19:21:11.977 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T19:21:12.382 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:12.650 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T19:21:12.779 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T19:21:12.943 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T19:21:13.065 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:13.222 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T19:21:13.288 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:14.020 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T19:21:14.287 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T19:21:14.495 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T19:21:15.286 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T19:21:33.096 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T19:21:33.096 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /dev 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /run 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /dev 2023-12-11T19:21:33.097 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:21:33.481 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2023-12-11T19:21:33.807 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-12-11T19:21:33.808 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:21:34.230 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T19:21:35.219 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T19:21:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T19:21:35.673 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T19:21:35.855 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T19:21:35.982 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T19:21:36.180 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /dev 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /run 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2023-12-11T19:21:36.328 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2023-12-11T19:21:36.329 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2023-12-11T19:21:36.329 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2023-12-11T19:21:36.329 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /dev 2023-12-11T19:21:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:21:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T19:21:36.466 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T19:21:36.571 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T19:21:36.613 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T19:21:36.646 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T19:21:36.734 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T19:21:37.169 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T19:21:37.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T19:21:37.447 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T19:21:38.607 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T19:21:38.776 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T19:21:39.064 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T19:21:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T19:21:39.541 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T19:21:39.689 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T19:21:39.799 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T19:21:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 103/135 2023-12-11T19:21:39.992 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T19:21:40.036 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T19:21:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 104/135 2023-12-11T19:21:40.270 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T19:21:40.379 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T19:21:40.538 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T19:21:40.550 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T19:21:40.696 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 103/135 2023-12-11T19:21:40.776 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T19:21:40.936 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 104/135 2023-12-11T19:21:41.153 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T19:21:41.426 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T19:21:41.621 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T19:21:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T19:21:42.055 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T19:21:42.181 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T19:21:42.530 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 108/135 2023-12-11T19:21:42.709 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T19:21:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T19:21:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T19:21:43.092 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T19:21:43.092 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T19:21:43.093 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T19:21:43.093 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T19:21:43.187 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T19:21:43.348 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T19:21:43.417 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 108/135 2023-12-11T19:21:43.484 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 103/135 2023-12-11T19:21:43.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T19:21:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 104/135 2023-12-11T19:21:43.737 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T19:21:43.858 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T19:21:43.986 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T19:21:44.015 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T19:21:44.140 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T19:21:44.441 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T19:21:44.538 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 111/135 2023-12-11T19:21:44.765 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T19:21:44.828 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T19:21:45.003 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T19:21:45.074 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T19:21:45.244 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T19:21:45.468 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 111/135 2023-12-11T19:21:45.601 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T19:21:45.662 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T19:21:45.701 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T19:21:45.765 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T19:21:45.791 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T19:21:45.861 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T19:21:45.926 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T19:21:45.986 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T19:21:46.058 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T19:21:46.245 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T19:21:46.298 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 108/135 2023-12-11T19:21:46.503 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T19:21:46.575 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T19:21:46.666 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T19:21:46.766 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T19:21:46.796 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T19:21:46.844 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T19:21:48.342 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 111/135 2023-12-11T19:21:48.538 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T19:21:48.603 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T19:21:48.774 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T19:21:48.849 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T19:21:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T19:21:49.429 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T19:21:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T19:21:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T19:21:51.912 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T19:21:52.128 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 116/135 2023-12-11T19:21:52.494 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 117/135 2023-12-11T19:21:52.681 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 118/135 2023-12-11T19:21:52.991 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/135 2023-12-11T19:21:53.176 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T19:21:53.193 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 116/135 2023-12-11T19:21:53.426 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T19:21:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 117/135 2023-12-11T19:21:53.681 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/135 2023-12-11T19:21:53.790 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 118/135 2023-12-11T19:21:53.856 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 122/135 2023-12-11T19:21:54.012 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T19:21:54.085 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/135 2023-12-11T19:21:54.259 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T19:21:54.525 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T19:21:54.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/135 2023-12-11T19:21:54.981 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T19:21:54.981 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 122/135 2023-12-11T19:21:55.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T19:21:55.259 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 116/135 2023-12-11T19:21:55.481 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/135 2023-12-11T19:21:55.754 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/135 2023-12-11T19:21:57.067 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/135 2023-12-11T19:21:57.421 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/135 2023-12-11T19:21:57.425 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:21:57.605 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 120/135 2023-12-11T19:21:57.727 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:21:57.765 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:21:57.837 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/135 2023-12-11T19:21:58.000 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 125/135 2023-12-11T19:21:58.012 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 122/135 2023-12-11T19:21:58.168 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T19:21:58.226 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 126/135 2023-12-11T19:21:58.576 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 127/135 2023-12-11T19:21:58.621 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:21:58.934 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:21:58.939 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-11T19:21:58.974 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:21:59.107 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T19:21:59.218 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 125/135 2023-12-11T19:21:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T19:21:59.432 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T19:21:59.461 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T19:21:59.561 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 126/135 2023-12-11T19:21:59.657 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T19:21:59.680 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T19:21:59.878 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 132/135 2023-12-11T19:21:59.878 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:21:59.928 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 127/135 2023-12-11T19:22:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T19:22:00.314 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-11T19:22:00.340 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T19:22:00.533 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T19:22:00.574 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:00.643 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T19:22:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:00.837 INFO:teuthology.orchestra.run.smithi070.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-11T19:22:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:22:00.859 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T19:22:00.887 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T19:22:01.125 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T19:22:01.149 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T19:22:01.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 132/135 2023-12-11T19:22:01.329 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:01.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/135 2023-12-11T19:22:01.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:22:01.460 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/135 2023-12-11T19:22:01.461 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-11T19:22:01.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-11T19:22:01.463 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 73/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 74/135 2023-12-11T19:22:01.464 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 76/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 77/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 78/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 80/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 83/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 84/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 85/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 87/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 89/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 90/135 2023-12-11T19:22:01.465 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 91/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 92/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 93/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 94/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 95/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 97/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 98/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 99/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 100/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 101/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 102/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 103/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/135 2023-12-11T19:22:01.466 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 106/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 107/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 108/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.1-7.el8.noarch 109/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 110/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 111/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.1-9.el8.x86_64 112/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 113/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 115/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 117/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 118/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 119/135 2023-12-11T19:22:01.467 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 120/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 121/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 122/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 124/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-3.el8.x86_64 125/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 126/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 127/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 128/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 129/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 130/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 132/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:01.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-11T19:22:01.582 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:01.582 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T19:22:01.810 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:22:01.890 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T19:22:02.048 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:22:02.088 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T19:22:02.142 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:02.349 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 125/135 2023-12-11T19:22:02.404 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:02.404 INFO:teuthology.orchestra.run.smithi167.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-11T19:22:02.404 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:22:02.558 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 126/135 2023-12-11T19:22:02.895 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 127/135 2023-12-11T19:22:03.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/135 2023-12-11T19:22:03.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/135 2023-12-11T19:22:03.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/135 2023-12-11T19:22:03.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/135 2023-12-11T19:22:03.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-11T19:22:03.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-11T19:22:03.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 73/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 74/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/135 2023-12-11T19:22:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 76/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 77/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 78/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 80/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 83/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 84/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 85/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 87/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 89/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 90/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 91/135 2023-12-11T19:22:03.053 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 92/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 93/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 94/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 95/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 97/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 98/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 99/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 100/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 101/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 102/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 103/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/135 2023-12-11T19:22:03.054 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 106/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 107/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 108/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.1-7.el8.noarch 109/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 110/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 111/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.1-9.el8.x86_64 112/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 113/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 115/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 117/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 118/135 2023-12-11T19:22:03.055 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 119/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 120/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 121/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 122/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 124/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.6-3.el8.x86_64 125/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 126/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 127/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 128/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 129/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 130/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 132/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:03.056 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-11T19:22:03.199 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-11T19:22:03.380 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-11T19:22:03.404 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-11T19:22:03.629 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 130/135 2023-12-11T19:22:03.813 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.1-9.el8.x86_64 131/135 2023-12-11T19:22:03.923 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 131/135 2023-12-11T19:22:04.114 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.0.4-10.el8.x86_64 132/135 2023-12-11T19:22:04.144 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 132/135 2023-12-11T19:22:04.144 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:04.395 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:04.396 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T19:22:04.658 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T19:22:04.848 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:05.113 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:05.113 INFO:teuthology.orchestra.run.smithi066.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-11T19:22:05.113 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:22:05.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/135 2023-12-11T19:22:05.783 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/135 2023-12-11T19:22:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/135 2023-12-11T19:22:05.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/135 2023-12-11T19:22:05.795 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-11T19:22:05.796 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-11T19:22:05.797 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 73/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 75/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 76/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 77/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 80/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 81/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 82/135 2023-12-11T19:22:05.798 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.1-7.el8.noarch 83/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 84/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 85/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.1-9.el8.x86_64 86/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 87/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 89/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 91/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 92/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 93/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 94/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 95/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 96/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/135 2023-12-11T19:22:05.799 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 98/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : jq-1.6-3.el8.x86_64 99/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 100/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 101/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 102/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 103/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 104/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 105/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 106/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 108/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 109/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 110/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 111/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/135 2023-12-11T19:22:05.800 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 114/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 115/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 116/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 117/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 118/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 119/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 121/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 122/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 124/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 125/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 126/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/135 2023-12-11T19:22:05.801 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 128/135 2023-12-11T19:22:05.802 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-11T19:22:05.802 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 130/135 2023-12-11T19:22:05.802 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-11T19:22:05.802 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 132/135 2023-12-11T19:22:05.802 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T19:22:05.802 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-11T19:22:06.784 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:06.784 INFO:teuthology.orchestra.run.smithi070.stdout:Installed products updated. 2023-12-11T19:22:07.034 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.1-7.el8.noarch 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.0.4-10.el8.x86_64 2023-12-11T19:22:07.035 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.036 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.1-9.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-11T19:22:07.037 INFO:teuthology.orchestra.run.smithi070.stdout: jq-1.6-3.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libpq-13.5-1.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-11T19:22:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-11T19:22:07.039 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-11T19:22:07.040 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-11T19:22:07.041 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-11T19:22:07.042 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-2.6-1.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-11T19:22:07.043 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.044 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-11T19:22:07.045 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-12-11T19:22:07.046 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-12-11T19:22:07.662 DEBUG:teuthology.parallel:result is None 2023-12-11T19:22:08.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:08.514 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.1-7.el8.noarch 2023-12-11T19:22:08.753 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.0.4-10.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.754 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.1-9.el8.x86_64 2023-12-11T19:22:08.755 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.6-3.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libpq-13.5-1.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.756 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-11T19:22:08.757 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:08.758 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-11T19:22:08.759 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-11T19:22:08.760 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-11T19:22:08.761 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:08.762 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:22:08.763 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-12-11T19:22:09.103 DEBUG:teuthology.parallel:result is None 2023-12-11T19:22:11.589 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T19:22:11.590 INFO:teuthology.orchestra.run.smithi066.stdout:Installed products updated. 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.1-7.el8.noarch 2023-12-11T19:22:11.839 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.0.4-10.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.1-9.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: jq-1.6-3.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-11T19:22:11.841 INFO:teuthology.orchestra.run.smithi066.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libpq-13.5-1.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.842 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-11T19:22:11.843 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-11T19:22:11.844 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-11T19:22:11.845 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-2.6-1.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-11T19:22:11.846 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-11T19:22:11.847 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-11T19:22:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-11T19:22:11.849 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-11T19:22:11.849 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:22:11.849 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-12-11T19:22:12.150 DEBUG:teuthology.parallel:result is None 2023-12-11T19:22:12.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:22:12.373 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-11T19:22:12.430 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-119.ga62fda73.el8 2023-12-11T19:22:12.431 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-11T19:22:12.431 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-11T19:22:12.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:22:12.646 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-11T19:22:12.706 INFO:teuthology.orchestra.run.smithi070.stdout:19.0.0-119.ga62fda73.el8 2023-12-11T19:22:12.707 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-11T19:22:12.707 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-11T19:22:12.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:22:12.927 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-11T19:22:12.986 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-119.ga62fda73.el8 2023-12-11T19:22:12.987 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-11T19:22:12.987 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-11T19:22:12.988 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-11T19:22:12.988 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:22:12.989 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-11T19:22:13.084 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:22:13.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-11T19:22:13.179 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:22:13.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-11T19:22:13.276 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-11T19:22:13.276 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:22:13.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2023-12-11T19:22:13.350 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-11T19:22:13.464 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:22:13.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2023-12-11T19:22:13.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-11T19:22:13.647 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:22:13.647 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2023-12-11T19:22:13.721 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-11T19:22:13.831 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-11T19:22:13.832 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:22:13.832 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-11T19:22:13.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-11T19:22:14.016 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:22:14.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-11T19:22:14.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-11T19:22:14.192 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:22:14.192 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-11T19:22:14.263 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-11T19:22:14.372 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-11T19:22:14.373 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:22:14.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2023-12-11T19:22:14.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-11T19:22:14.557 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:22:14.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2023-12-11T19:22:14.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-11T19:22:14.732 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:22:14.733 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2023-12-11T19:22:14.801 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-11T19:22:14.910 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-11T19:22:15.003 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'subvols': {'create': 2}} 2023-12-11T19:22:15.004 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:22:15.004 INFO:tasks.cephadm:Cluster fsid is 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:22:15.004 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-11T19:22:15.004 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.66', 'mon.b': '172.21.15.70', 'mon.c': '172.21.15.167'} 2023-12-11T19:22:15.005 INFO:tasks.cephadm:First mon is mon.a on smithi066 2023-12-11T19:22:15.005 INFO:tasks.cephadm:First mgr is x 2023-12-11T19:22:15.005 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-11T19:22:15.005 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2023-12-11T19:22:15.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2023-12-11T19:22:15.149 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2023-12-11T19:22:15.219 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-11T19:22:15.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:22:15.437 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74860/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-119-ga62fda73', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-119.ga62fda73'}, 'url': 'https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 13:31:01.711792', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023dec11', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74860/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-119-ga62fda73', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-119.ga62fda73'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 12:43:31.974707', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023dec11', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-12-11T19:22:15.573 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref rishabh-2023dec11, sha1 a62fda73efba5520d409bb5a21bd8373289dda5d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:22:15.575 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm 2023-12-11T19:22:15.575 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:16.483 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 11 19:22 /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:16.484 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:17.094 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 11 19:22 /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:17.094 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:17.997 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 11 19:22 /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:17.998 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:18.051 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:18.101 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-11T19:22:18.168 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d on all hosts... 2023-12-11T19:22:18.168 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-11T19:22:18.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-11T19:22:18.178 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-11T19:22:22.174 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T19:22:22.739 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T19:22:22.804 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2023-12-11T19:22:54.209 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-12-11T19:22:55.957 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-12-11T19:22:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T19:22:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-11T19:22:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2023-12-11T19:22:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-11T19:22:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-12-11T19:22:55.958 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-12-11T19:22:57.590 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:22:57.591 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T19:22:57.591 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-11T19:22:57.591 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2023-12-11T19:22:57.591 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-11T19:22:57.591 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:22:57.591 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:22:57.624 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2023-12-11T19:22:57.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2023-12-11T19:22:57.778 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2023-12-11T19:22:57.858 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2023-12-11T19:22:57.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2023-12-11T19:22:58.001 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2023-12-11T19:22:58.073 INFO:tasks.cephadm:Writing seed config... 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-11T19:22:58.074 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-11T19:22:58.075 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-11T19:22:58.076 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-11T19:22:58.077 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-11T19:22:58.077 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-11T19:22:58.077 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-12-11T19:22:58.077 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:22:58.078 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-11T19:22:58.122 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 976cf7f8-985a-11ee-95a3-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-11T19:22:58.122 DEBUG:teuthology.orchestra.run.smithi066:mon.a> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a.service 2023-12-11T19:22:58.166 DEBUG:teuthology.orchestra.run.smithi066:mgr.x> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x.service 2023-12-11T19:22:58.211 INFO:tasks.cephadm:Bootstrapping... 2023-12-11T19:22:58.212 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d -v bootstrap --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.66 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:22:58.236 INFO:journalctl@ceph.mon.a.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:22:58.281 INFO:journalctl@ceph.mgr.x.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:22:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2023-12-11T19:22:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d', '-v', 'bootstrap', '--fsid', '976cf7f8-985a-11ee-95a3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.66', '--skip-admin-label'] 2023-12-11T19:22:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.1.1 2023-12-11T19:22:58.674 INFO:teuthology.orchestra.run.smithi066.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-11T19:22:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying podman|docker is present... 2023-12-11T19:22:58.849 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.1.1 2023-12-11T19:22:58.849 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying lvm2 is present... 2023-12-11T19:22:58.850 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying time synchronization is in place... 2023-12-11T19:22:58.865 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-11T19:22:58.865 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-11T19:22:58.878 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-11T19:22:58.878 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-12-11T19:22:58.892 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2023-12-11T19:22:58.906 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2023-12-11T19:22:58.906 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2023-12-11T19:22:58.906 INFO:teuthology.orchestra.run.smithi066.stdout:Repeating the final host check... 2023-12-11T19:22:59.081 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.1.1 2023-12-11T19:22:59.081 INFO:teuthology.orchestra.run.smithi066.stdout:podman (/bin/podman) version 4.1.1 is present 2023-12-11T19:22:59.081 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl is present 2023-12-11T19:22:59.081 INFO:teuthology.orchestra.run.smithi066.stdout:lvcreate is present 2023-12-11T19:22:59.094 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-11T19:22:59.094 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-11T19:22:59.108 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-11T19:22:59.108 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-12-11T19:22:59.122 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2023-12-11T19:22:59.135 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2023-12-11T19:22:59.135 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2023-12-11T19:22:59.136 INFO:teuthology.orchestra.run.smithi066.stdout:Host looks OK 2023-12-11T19:22:59.136 INFO:teuthology.orchestra.run.smithi066.stdout:Cluster fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:22:59.136 INFO:teuthology.orchestra.run.smithi066.stdout:Acquiring lock 140191892901392 on /run/cephadm/976cf7f8-985a-11ee-95a3-87774f69a715.lock 2023-12-11T19:22:59.136 INFO:teuthology.orchestra.run.smithi066.stdout:Lock 140191892901392 acquired on /run/cephadm/976cf7f8-985a-11ee-95a3-87774f69a715.lock 2023-12-11T19:22:59.136 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 3300 ... 2023-12-11T19:22:59.137 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 6789 ... 2023-12-11T19:22:59.137 INFO:teuthology.orchestra.run.smithi066.stdout:Base mon IP(s) is [172.21.15.66:3300, 172.21.15.66:6789], mon addrv is [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2023-12-11T19:22:59.145 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-12-11T19:22:59.145 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.66 metric 100 2023-12-11T19:22:59.152 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-11T19:22:59.152 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-12-11T19:22:59.152 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-12-11T19:22:59.159 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-11T19:22:59.159 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-11T19:22:59.159 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-11T19:22:59.159 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-12-11T19:22:59.159 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 fe80::6e70:b1a2:cd6d:66fc/64 scope link noprefixroute 2023-12-11T19:22:59.159 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-11T19:22:59.160 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2023-12-11T19:22:59.160 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2023-12-11T19:22:59.160 INFO:teuthology.orchestra.run.smithi066.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-11T19:22:59.161 INFO:teuthology.orchestra.run.smithi066.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-11T19:22:59.162 INFO:teuthology.orchestra.run.smithi066.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T19:22:59.641 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a 2023-12-11T19:22:59.641 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T19:22:59.641 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Getting image source signatures 2023-12-11T19:22:59.641 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:933954f0af8f706d6a6186e87e0ea462c846b5abc6ef29ee9ef20dac46eff0f7 2023-12-11T19:22:59.642 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:d3e436ac93e4b5a6d94c7ac8024df331e66e67275836276615dc044fd3cbf83a 2023-12-11T19:22:59.642 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying config sha256:0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a 2023-12-11T19:22:59.642 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-11T19:22:59.642 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Storing signatures 2023-12-11T19:23:00.621 INFO:teuthology.orchestra.run.smithi066.stdout:ceph: stdout ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev) 2023-12-11T19:23:00.621 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph version: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev) 2023-12-11T19:23:00.621 INFO:teuthology.orchestra.run.smithi066.stdout:Extracting ceph user uid/gid from container image... 2023-12-11T19:23:01.504 INFO:teuthology.orchestra.run.smithi066.stdout:stat: stdout 167 167 2023-12-11T19:23:01.504 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial keys... 2023-12-11T19:23:02.785 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQCVYXdlRgr1NxAAtLl1eGGrza3/9+vm2f+lXQ== 2023-12-11T19:23:03.670 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQCXYXdlb4k2CxAAFo9JVzuwOuMh4zp3YqXBzA== 2023-12-11T19:23:04.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQCYYXdlP/7BBhAAL6tW3gD8OVE2eSlm/E3SZg== 2023-12-11T19:23:04.448 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial monmap... 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool for a [v2:172.21.15.66:3300,v1:172.21.15.66:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = pacific 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: set fsid to 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-11T19:23:05.232 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:23:05.233 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mon... 2023-12-11T19:23:06.113 INFO:teuthology.orchestra.run.smithi066.stdout:create mon.a on 2023-12-11T19:23:06.368 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-11T19:23:06.580 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-11T19:23:06.831 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-976cf7f8-985a-11ee-95a3-87774f69a715.target → /etc/systemd/system/ceph-976cf7f8-985a-11ee-95a3-87774f69a715.target. 2023-12-11T19:23:06.831 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-976cf7f8-985a-11ee-95a3-87774f69a715.target → /etc/systemd/system/ceph-976cf7f8-985a-11ee-95a3-87774f69a715.target. 2023-12-11T19:23:07.203 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a 2023-12-11T19:23:07.203 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a.service: Unit ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a.service not loaded. 2023-12-11T19:23:07.421 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-976cf7f8-985a-11ee-95a3-87774f69a715.target.wants/ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-976cf7f8-985a-11ee-95a3-87774f69a715@.service. 2023-12-11T19:23:07.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:07 smithi066 systemd[1]: Starting Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:23:09.131 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:08 smithi066 podman[97372]: 2023-12-11T19:23:09.131 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:09 smithi066 bash[97372]: 57f63b54a5d5c07a40e230bb0912c79c7ddc7ac944e27cef03d069ee8b87226b 2023-12-11T19:23:09.157 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T19:23:09.157 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-12-11T19:23:09.174 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T19:23:09.175 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-12-11T19:23:09.175 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-12-11T19:23:09.175 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2023-12-11T19:23:09.175 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon to start... 2023-12-11T19:23:09.175 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon... 2023-12-11T19:23:09.430 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:09 smithi066 systemd[1]: Started Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:23:09.430 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:09 smithi066 ceph-mon[97412]: mkfs 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:09.430 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:09 smithi066 ceph-mon[97412]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout cluster: 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout id: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout services: 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.652925s) 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:10.268 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout data: 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pgs: 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.707+0000 7f8fdee06700 1 Processor -- start 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.707+0000 7f8fdee06700 1 -- start start 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.707+0000 7f8fdee06700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.707+0000 7f8fdee06700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd807ae70 con 0x7f8fd8105370 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.708+0000 7f8fdcba2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.708+0000 7f8fdcba2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54620/0 (socket says 172.21.15.66:54620) 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.708+0000 7f8fdcba2700 1 -- 172.21.15.66:0/3408832017 learned_addr learned my addr 172.21.15.66:0/3408832017 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.709+0000 7f8fdcba2700 1 -- 172.21.15.66:0/3408832017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8107ce0 con 0x7f8fd8105370 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.709+0000 7f8fdcba2700 1 --2- 172.21.15.66:0/3408832017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81077a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0098a0 tx=0x7f8fcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b904c776efc9cd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.709+0000 7f8fd77fe700 1 -- 172.21.15.66:0/3408832017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fcc004160 con 0x7f8fd8105370 2023-12-11T19:23:10.269 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.709+0000 7f8fd77fe700 1 -- 172.21.15.66:0/3408832017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc0042c0 con 0x7f8fd8105370 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.709+0000 7f8fd77fe700 1 -- 172.21.15.66:0/3408832017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fcc0045a0 con 0x7f8fd8105370 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 -- 172.21.15.66:0/3408832017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 msgr2=0x7f8fd81077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 --2- 172.21.15.66:0/3408832017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81077a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0098a0 tx=0x7f8fcc009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 -- 172.21.15.66:0/3408832017 shutdown_connections 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 --2- 172.21.15.66:0/3408832017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81077a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 -- 172.21.15.66:0/3408832017 >> 172.21.15.66:0/3408832017 conn(0x7f8fd8101010 msgr2=0x7f8fd8103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 -- 172.21.15.66:0/3408832017 shutdown_connections 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.710+0000 7f8fdee06700 1 -- 172.21.15.66:0/3408832017 wait complete. 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.711+0000 7f8fdee06700 1 Processor -- start 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.711+0000 7f8fdee06700 1 -- start start 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.711+0000 7f8fdee06700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.711+0000 7f8fdee06700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd81a0ba0 con 0x7f8fd8105370 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.712+0000 7f8fdcba2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:10.270 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.712+0000 7f8fdcba2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54636/0 (socket says 172.21.15.66:54636) 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.712+0000 7f8fdcba2700 1 -- 172.21.15.66:0/4252872438 learned_addr learned my addr 172.21.15.66:0/4252872438 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.712+0000 7f8fdcba2700 1 -- 172.21.15.66:0/4252872438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc009550 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.712+0000 7f8fdcba2700 1 --2- 172.21.15.66:0/4252872438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81a0660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8fd807a2b0 tx=0x7f8fcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fcc011630 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc011790 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fcc011a30 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd81a0da0 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd81a1240 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8fcc022070 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.713+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8fcc01c450 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.714+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd819a650 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.717+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8fcc01b450 con 0x7f8fd8105370 2023-12-11T19:23:10.271 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.795+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8fd8108280 con 0x7f8fd8105370 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.795+0000 7f8fd5ffb700 1 -- 172.21.15.66:0/4252872438 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f8fcc01b5f0 con 0x7f8fd8105370 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 msgr2=0x7f8fd81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 --2- 172.21.15.66:0/4252872438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81a0660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8fd807a2b0 tx=0x7f8fcc004750 comp rx=0 tx=0).stop 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 shutdown_connections 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 --2- 172.21.15.66:0/4252872438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fd8105370 0x7f8fd81a0660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 >> 172.21.15.66:0/4252872438 conn(0x7f8fd8101010 msgr2=0x7f8fd81032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 shutdown_connections 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:09.796+0000 7f8fdee06700 1 -- 172.21.15.66:0/4252872438 wait complete. 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:mon is available 2023-12-11T19:23:10.272 INFO:teuthology.orchestra.run.smithi066.stdout:Assimilating anything we can from ceph.conf... 2023-12-11T19:23:10.547 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:10 smithi066 ceph-mon[97412]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-11T19:23:10.547 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:10 smithi066 ceph-mon[97412]: monmap e1: 1 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:23:10.548 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:10 smithi066 ceph-mon[97412]: fsmap 2023-12-11T19:23:10.548 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:10 smithi066 ceph-mon[97412]: osdmap e1: 0 total, 0 up, 0 in 2023-12-11T19:23:10.548 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:10 smithi066 ceph-mon[97412]: mgrmap e1: no daemons active 2023-12-11T19:23:10.548 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:10 smithi066 ceph-mon[97412]: from='client.? 172.21.15.66:0/4252872438' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-11T19:23:11.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:11 smithi066 ceph-mon[97412]: from='client.? 172.21.15.66:0/2150680227' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-11T19:23:11.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:11 smithi066 ceph-mon[97412]: from='client.? 172.21.15.66:0/2150680227' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-11T19:23:11.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.815+0000 7fe790c69700 1 Processor -- start 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.815+0000 7fe790c69700 1 -- start start 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.816+0000 7fe790c69700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c07b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.816+0000 7fe790c69700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c07b870 con 0x7fe78c07ce40 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.816+0000 7fe78a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c07b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.816+0000 7fe78a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c07b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54646/0 (socket says 172.21.15.66:54646) 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.816+0000 7fe78a59c700 1 -- 172.21.15.66:0/3886402704 learned_addr learned my addr 172.21.15.66:0/3886402704 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:11.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.816+0000 7fe78a59c700 1 -- 172.21.15.66:0/3886402704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c07b9b0 con 0x7fe78c07ce40 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe78a59c700 1 --2- 172.21.15.66:0/3886402704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c07b2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098a0 tx=0x7fe77c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7bd961260d90d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe78959a700 1 -- 172.21.15.66:0/3886402704 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe77c004160 con 0x7fe78c07ce40 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe78959a700 1 -- 172.21.15.66:0/3886402704 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c0042c0 con 0x7fe78c07ce40 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe78959a700 1 -- 172.21.15.66:0/3886402704 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe77c0045a0 con 0x7fe78c07ce40 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe790c69700 1 -- 172.21.15.66:0/3886402704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 msgr2=0x7fe78c07b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe790c69700 1 --2- 172.21.15.66:0/3886402704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c07b2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098a0 tx=0x7fe77c009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe790c69700 1 -- 172.21.15.66:0/3886402704 shutdown_connections 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe790c69700 1 --2- 172.21.15.66:0/3886402704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c07b2a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.817+0000 7fe790c69700 1 -- 172.21.15.66:0/3886402704 >> 172.21.15.66:0/3886402704 conn(0x7fe78c1020c0 msgr2=0x7fe78c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.818+0000 7fe790c69700 1 -- 172.21.15.66:0/3886402704 shutdown_connections 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.818+0000 7fe790c69700 1 -- 172.21.15.66:0/3886402704 wait complete. 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.819+0000 7fe790c69700 1 Processor -- start 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.819+0000 7fe790c69700 1 -- start start 2023-12-11T19:23:11.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.819+0000 7fe790c69700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.819+0000 7fe790c69700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c19c750 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe78a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe78a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c19c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54660/0 (socket says 172.21.15.66:54660) 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe78a59c700 1 -- 172.21.15.66:0/2150680227 learned_addr learned my addr 172.21.15.66:0/2150680227 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe78a59c700 1 -- 172.21.15.66:0/2150680227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009550 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe78a59c700 1 --2- 172.21.15.66:0/2150680227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c19c210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe78c07c310 tx=0x7fe77c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe77c011630 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.820+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c011790 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.821+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c19c950 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.821+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe77c011a30 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.821+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c19cdf0 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.821+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe77c022070 con 0x7fe78c07ce40 2023-12-11T19:23:11.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.821+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe77c01c450 con 0x7fe78c07ce40 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.822+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c196080 con 0x7fe78c07ce40 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.824+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe77c01b450 con 0x7fe78c07ce40 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.899+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe78c07bf90 con 0x7fe78c07ce40 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.904+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fe77c01b5f0 con 0x7fe78c07ce40 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.904+0000 7fe77b7fe700 1 -- 172.21.15.66:0/2150680227 <== mon.0 v2:172.21.15.66:3300/0 8 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe77c01b830 con 0x7fe78c07ce40 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.905+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 msgr2=0x7fe78c19c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.905+0000 7fe790c69700 1 --2- 172.21.15.66:0/2150680227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c19c210 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe78c07c310 tx=0x7fe77c004750 comp rx=0 tx=0).stop 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.905+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 shutdown_connections 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.905+0000 7fe790c69700 1 --2- 172.21.15.66:0/2150680227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe78c07ce40 0x7fe78c19c210 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.905+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 >> 172.21.15.66:0/2150680227 conn(0x7fe78c1020c0 msgr2=0x7fe78c102530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.905+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 shutdown_connections 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:10.906+0000 7fe790c69700 1 -- 172.21.15.66:0/2150680227 wait complete. 2023-12-11T19:23:11.359 INFO:teuthology.orchestra.run.smithi066.stdout:Generating new minimal ceph.conf... 2023-12-11T19:23:12.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:12 smithi066 ceph-mon[97412]: from='client.? 172.21.15.66:0/2676668245' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:23:12.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.954+0000 7f977a4ee700 1 Processor -- start 2023-12-11T19:23:12.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.955+0000 7f977a4ee700 1 -- start start 2023-12-11T19:23:12.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.955+0000 7f977a4ee700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f9774106f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.955+0000 7f977a4ee700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774107550 con 0x7f9774106b60 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.955+0000 7f9773fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f9774106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.955+0000 7f9773fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f9774106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54664/0 (socket says 172.21.15.66:54664) 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.955+0000 7f9773fff700 1 -- 172.21.15.66:0/3324970390 learned_addr learned my addr 172.21.15.66:0/3324970390 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.956+0000 7f9773fff700 1 -- 172.21.15.66:0/3324970390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774107d60 con 0x7f9774106b60 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.956+0000 7f9773fff700 1 --2- 172.21.15.66:0/3324970390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f9774106f80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f97640098a0 tx=0x7f9764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bea20ba00a2696 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.956+0000 7f9772ffd700 1 -- 172.21.15.66:0/3324970390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9764004160 con 0x7f9774106b60 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.956+0000 7f9772ffd700 1 -- 172.21.15.66:0/3324970390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f97640042c0 con 0x7f9774106b60 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.956+0000 7f9772ffd700 1 -- 172.21.15.66:0/3324970390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9764011940 con 0x7f9774106b60 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 -- 172.21.15.66:0/3324970390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 msgr2=0x7f9774106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 --2- 172.21.15.66:0/3324970390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f9774106f80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f97640098a0 tx=0x7f9764009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 -- 172.21.15.66:0/3324970390 shutdown_connections 2023-12-11T19:23:12.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 --2- 172.21.15.66:0/3324970390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f9774106f80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 -- 172.21.15.66:0/3324970390 >> 172.21.15.66:0/3324970390 conn(0x7f97741020c0 msgr2=0x7f9774104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 -- 172.21.15.66:0/3324970390 shutdown_connections 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.957+0000 7f977a4ee700 1 -- 172.21.15.66:0/3324970390 wait complete. 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.959+0000 7f977a4ee700 1 Processor -- start 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.959+0000 7f977a4ee700 1 -- start start 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.959+0000 7f977a4ee700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f977407cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.959+0000 7f977a4ee700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977407cff0 con 0x7f9774106b60 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.960+0000 7f9773fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f977407cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.960+0000 7f9773fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f977407cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54666/0 (socket says 172.21.15.66:54666) 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.960+0000 7f9773fff700 1 -- 172.21.15.66:0/2676668245 learned_addr learned my addr 172.21.15.66:0/2676668245 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.960+0000 7f9773fff700 1 -- 172.21.15.66:0/2676668245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764009550 con 0x7f9774106b60 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.960+0000 7f9773fff700 1 --2- 172.21.15.66:0/2676668245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f977407cab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9764003f20 tx=0x7f9764011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.960+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97640036a0 con 0x7f9774106b60 2023-12-11T19:23:12.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.961+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f976401b440 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.961+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977407b280 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.961+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9764023400 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.961+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977407b720 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.961+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9764023970 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.961+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f976402cdf0 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.962+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9774062c00 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:11.965+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9764023d60 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.039+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f977410af90 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.039+0000 7f97717fa700 1 -- 172.21.15.66:0/2676668245 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f976401b5b0 con 0x7f9774106b60 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.040+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 msgr2=0x7f977407cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.040+0000 7f977a4ee700 1 --2- 172.21.15.66:0/2676668245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f977407cab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9764003f20 tx=0x7f9764011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.041+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 shutdown_connections 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.041+0000 7f977a4ee700 1 --2- 172.21.15.66:0/2676668245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9774106b60 0x7f977407cab0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:12.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.041+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 >> 172.21.15.66:0/2676668245 conn(0x7f97741020c0 msgr2=0x7f9774102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:12.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.041+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 shutdown_connections 2023-12-11T19:23:12.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:12.041+0000 7f977a4ee700 1 -- 172.21.15.66:0/2676668245 wait complete. 2023-12-11T19:23:12.543 INFO:teuthology.orchestra.run.smithi066.stdout:Restarting the monitor... 2023-12-11T19:23:12.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:12 smithi066 systemd[1]: Stopping Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:23:13.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a[97388]: 2023-12-11T19:23:13.002+0000 7fe150bdc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:23:13.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a[97388]: 2023-12-11T19:23:13.002+0000 7fe150bdc700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-11T19:23:13.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:13 smithi066 bash[97752]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a 2023-12-11T19:23:13.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:13 smithi066 systemd[1]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a.service: Succeeded. 2023-12-11T19:23:13.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:13 smithi066 systemd[1]: Stopped Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:23:13.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:13 smithi066 systemd[1]: Starting Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:23:14.686 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 podman[97880]: 2023-12-11T19:23:14.687 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 bash[97880]: 6739963f110ca4f2674f665a85c276ebae7e1e658e9ff5384282a9fc09d418fa 2023-12-11T19:23:14.687 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: set uid:gid to 167:167 (ceph:ceph) 2023-12-11T19:23:14.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: pidfile_write: ignore empty --pid-file 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: load: jerasure load: lrc 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: RocksDB version: 7.9.2 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Git sha 0 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: DB SUMMARY 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: DB Session ID: 8F83N13F144IEFYBZRO8 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: CURRENT file: CURRENT 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: IDENTITY file: IDENTITY 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-11T19:23:14.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98455 ; 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.error_if_exists: 0 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.create_if_missing: 0 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.paranoid_checks: 1 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.env: 0x55aaa2bcaa60 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.fs: PosixFileSystem 2023-12-11T19:23:14.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.info_log: 0x55aaa38ed780 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_file_opening_threads: 16 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.statistics: (nil) 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.use_fsync: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_log_file_size: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.keep_log_file_num: 1000 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.recycle_log_file_num: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.allow_fallocate: 1 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.allow_mmap_reads: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.allow_mmap_writes: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.use_direct_reads: 0 2023-12-11T19:23:14.711 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-11T19:23:14.753 INFO:teuthology.orchestra.run.smithi066.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-12-11T19:23:15.020 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.create_missing_column_families: 0 2023-12-11T19:23:15.020 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.db_log_dir: 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.wal_dir: 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-11T19:23:15.021 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.advise_random_on_open: 1 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.db_write_buffer_size: 0 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.write_buffer_manager: 0x55aaa38a3360 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.rate_limiter: (nil) 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-11T19:23:15.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.wal_recovery_mode: 2 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.enable_thread_tracking: 0 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.enable_pipelined_write: 0 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.unordered_write: 0 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.row_cache: None 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.wal_filter: None 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-11T19:23:15.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.allow_ingest_behind: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.two_write_queues: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.manual_wal_flush: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.wal_compression: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.atomic_flush: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.log_readahead_size: 0 2023-12-11T19:23:15.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-11T19:23:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.best_efforts_recovery: 0 2023-12-11T19:23:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-11T19:23:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-11T19:23:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.allow_data_in_errors: 0 2023-12-11T19:23:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.db_host_id: __hostname__ 2023-12-11T19:23:15.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_background_jobs: 2 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_background_compactions: -1 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_subcompactions: 1 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_total_wal_size: 0 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_open_files: -1 2023-12-11T19:23:15.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bytes_per_sync: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_readahead_size: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_background_flushes: -1 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Compression algorithms supported: 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kZSTD supported: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kXpressCompression supported: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kLZ4HCCompression supported: 1 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kZlibCompression supported: 1 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kSnappyCompression supported: 1 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kLZ4Compression supported: 1 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: kBZip2Compression supported: 0 2023-12-11T19:23:15.027 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.merge_operator: 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_filter: None 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_filter_factory: None 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.sst_partitioner_factory: None 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55aaa38ed8a0) 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks: 1 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-11T19:23:15.028 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_top_level_index_and_filter: 1 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_type: 0 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_index_type: 0 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_shortening: 1 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: checksum: 4 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: no_block_cache: 0 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache: 0x55aaa3943090 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_name: BinnedLRUCache 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_options: 2023-12-11T19:23:15.029 INFO:journalctl@ceph.mon.a.smithi066.stdout: capacity : 536870912 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_shard_bits : 4 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: strict_capacity_limit : 0 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: high_pri_pool_ratio: 0.000 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_compressed: (nil) 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: persistent_cache: (nil) 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size: 4096 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size_deviation: 10 2023-12-11T19:23:15.030 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_restart_interval: 16 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_block_restart_interval: 1 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: metadata_block_size: 4096 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: partition_filters: 0 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: use_delta_encoding: 1 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: filter_policy: bloomfilter 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: whole_key_filtering: 1 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: verify_compression: 0 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: read_amp_bytes_per_bit: 0 2023-12-11T19:23:15.031 INFO:journalctl@ceph.mon.a.smithi066.stdout: format_version: 5 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_index_compression: 1 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_align: 0 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout: max_auto_readahead_size: 262144 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout: prepopulate_block_cache: 0 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout: initial_auto_readahead_size: 8192 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_file_reads_for_auto_readahead: 2 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.write_buffer_size: 33554432 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_write_buffer_number: 2 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression: NoCompression 2023-12-11T19:23:15.032 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression: Disabled 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.prefix_extractor: nullptr 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.num_levels: 7 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-11T19:23:15.033 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.level: 32767 2023-12-11T19:23:15.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.strategy: 0 2023-12-11T19:23:15.035 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-11T19:23:15.035 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-11T19:23:15.035 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-11T19:23:15.035 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-11T19:23:15.035 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.enabled: false 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.target_file_size_base: 67108864 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-11T19:23:15.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-11T19:23:15.037 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.arena_block_size: 1048576 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.disable_auto_compactions: 0 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-11T19:23:15.038 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.table_properties_collectors: 2023-12-11T19:23:15.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.inplace_update_support: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.bloom_locality: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.max_successive_merges: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.paranoid_file_checks: 0 2023-12-11T19:23:15.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.force_consistency_checks: 1 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.report_bg_io_stats: 0 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.ttl: 2592000 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.enable_blob_files: false 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.min_blob_size: 0 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.blob_file_size: 268435456 2023-12-11T19:23:15.041 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.blob_file_starting_level: 0 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-11T19:23:15.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: af0dabc5-5786-4bdf-9598-1dc811b3e888 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322594699438, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322594702004, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93750, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91750, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13527, "raw_average_key_size": 49, "raw_value_size": 84369, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702322594, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "af0dabc5-5786-4bdf-9598-1dc811b3e888", "db_session_id": "8F83N13F144IEFYBZRO8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322594702213, "job": 1, "event": "recovery_finished"} 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55aaa39de000 2023-12-11T19:23:15.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: DB pointer 0x55aaa39ca000 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** DB Stats ** 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T19:23:15.044 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: L0 2/0 93.39 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: Sum 2/0 93.39 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-11T19:23:15.045 INFO:journalctl@ceph.mon.a.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 64.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-11T19:23:15.046 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative compaction: 0.00 GB write, 9.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval compaction: 0.00 GB write, 9.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: Block cache BinnedLRUCache@0x55aaa3943090#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: starting mon.a rank 0 at public addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] at bind addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???) e1 preinit fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).mds e1 new map 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).mds e1 print_map 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: e1 2023-12-11T19:23:15.047 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout: legacy client fscid: -1 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout: No filesystems configured 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 systemd[1]: Started Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: monmap e1: 1 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: fsmap 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: osdmap e1: 0 total, 0 up, 0 in 2023-12-11T19:23:15.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:14 smithi066 ceph-mon[97920]: mgrmap e1: no daemons active 2023-12-11T19:23:15.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.285+0000 7ff530a26700 1 Processor -- start 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.285+0000 7ff530a26700 1 -- start start 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.286+0000 7ff530a26700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.286+0000 7ff530a26700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c07ae70 con 0x7ff52c105370 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.286+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.286+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33770/0 (socket says 172.21.15.66:33770) 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.286+0000 7ff52a59c700 1 -- 172.21.15.66:0/1941766260 learned_addr learned my addr 172.21.15.66:0/1941766260 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.287+0000 7ff52a59c700 1 -- 172.21.15.66:0/1941766260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c107ce0 con 0x7ff52c105370 2023-12-11T19:23:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.287+0000 7ff52a59c700 1 --2- 172.21.15.66:0/1941766260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1077a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0098a0 tx=0x7ff51c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d52cc0918589973 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:15.741 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.287+0000 7ff52959a700 1 -- 172.21.15.66:0/1941766260 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff51c004160 con 0x7ff52c105370 2023-12-11T19:23:15.743 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.287+0000 7ff52959a700 1 -- 172.21.15.66:0/1941766260 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff51c0042c0 con 0x7ff52c105370 2023-12-11T19:23:15.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.287+0000 7ff52959a700 1 -- 172.21.15.66:0/1941766260 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff51c011940 con 0x7ff52c105370 2023-12-11T19:23:15.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 -- 172.21.15.66:0/1941766260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 msgr2=0x7ff52c1077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:15.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 --2- 172.21.15.66:0/1941766260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1077a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0098a0 tx=0x7ff51c009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:15.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 -- 172.21.15.66:0/1941766260 shutdown_connections 2023-12-11T19:23:15.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 --2- 172.21.15.66:0/1941766260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1077a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:15.744 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 -- 172.21.15.66:0/1941766260 >> 172.21.15.66:0/1941766260 conn(0x7ff52c101010 msgr2=0x7ff52c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:15.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 -- 172.21.15.66:0/1941766260 shutdown_connections 2023-12-11T19:23:15.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.288+0000 7ff530a26700 1 -- 172.21.15.66:0/1941766260 wait complete. 2023-12-11T19:23:15.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.289+0000 7ff530a26700 1 Processor -- start 2023-12-11T19:23:15.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.289+0000 7ff530a26700 1 -- start start 2023-12-11T19:23:15.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.290+0000 7ff530a26700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:15.745 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.290+0000 7ff530a26700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1a12b0 con 0x7ff52c105370 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.290+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.290+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33772/0 (socket says 172.21.15.66:33772) 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.290+0000 7ff52a59c700 1 -- 172.21.15.66:0/2058774192 learned_addr learned my addr 172.21.15.66:0/2058774192 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.290+0000 7ff52a59c700 1 -- 172.21.15.66:0/2058774192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c009550 con 0x7ff52c105370 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff52a59c700 1 --2- 172.21.15.66:0/2058774192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1a0d70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff51c003f20 tx=0x7ff51c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff51c0037d0 con 0x7ff52c105370 2023-12-11T19:23:15.746 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff51c01b440 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff51c023590 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c1a14b0 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c1a1950 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.291+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff51c023b00 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.292+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff51c02cb00 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.292+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c19a650 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.295+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff51c01b5b0 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.369+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ff52c108da0 con 0x7ff52c105370 2023-12-11T19:23:15.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.371+0000 7ff51b7fe700 1 -- 172.21.15.66:0/2058774192 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff51c01b920 con 0x7ff52c105370 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.372+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 msgr2=0x7ff52c1a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.372+0000 7ff530a26700 1 --2- 172.21.15.66:0/2058774192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1a0d70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff51c003f20 tx=0x7ff51c011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.372+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 shutdown_connections 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.372+0000 7ff530a26700 1 --2- 172.21.15.66:0/2058774192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c105370 0x7ff52c1a0d70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.372+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 >> 172.21.15.66:0/2058774192 conn(0x7ff52c101010 msgr2=0x7ff52c103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.373+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 shutdown_connections 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:15.373+0000 7ff530a26700 1 -- 172.21.15.66:0/2058774192 wait complete. 2023-12-11T19:23:15.748 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-11T19:23:15.770 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:23:15.771 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mgr... 2023-12-11T19:23:15.771 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-11T19:23:15.772 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-11T19:23:16.137 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x 2023-12-11T19:23:16.137 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x.service: Unit ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x.service not loaded. 2023-12-11T19:23:16.351 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-976cf7f8-985a-11ee-95a3-87774f69a715.target.wants/ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-976cf7f8-985a-11ee-95a3-87774f69a715@.service. 2023-12-11T19:23:16.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:16 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2058774192' entity='client.admin' 2023-12-11T19:23:16.564 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:16 smithi066 systemd[1]: Starting Ceph mgr.x for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:23:17.166 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:16 smithi066 podman[98153]: 2023-12-11T19:23:17.166 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 bash[98153]: 227ea30622be7c6001c0e770bb833b512c4a8c7614bf6a6c86ea268df7a0096a 2023-12-11T19:23:17.195 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T19:23:17.195 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-12-11T19:23:17.215 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T19:23:17.215 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-12-11T19:23:17.215 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-12-11T19:23:17.215 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2023-12-11T19:23:17.237 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T19:23:17.237 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-12-11T19:23:17.255 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T19:23:17.255 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-12-11T19:23:17.255 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-12-11T19:23:17.256 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-11T19:23:17.256 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr to start... 2023-12-11T19:23:17.256 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr... 2023-12-11T19:23:17.420 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 systemd[1]: Started Ceph mgr.x for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:23:17.420 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:17.295+0000 7f6236970200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T19:23:17.681 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:17.419+0000 7f6236970200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T19:23:17.681 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:17.547+0000 7f6236970200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T19:23:17.961 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:17.680+0000 7f6236970200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T19:23:17.961 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:17.922+0000 7f6236970200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T19:23:18.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:17 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2742002669' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-11T19:23:18.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:18.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:23:18.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:23:18.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-11T19:23:18.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-11T19:23:18.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-11T19:23:18.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:18.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-11T19:23:18.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-11T19:23:18.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-11T19:23:18.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2023-12-11T19:23:09.144924+0000", 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.848+0000 7f9df6ca7700 1 Processor -- start 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.849+0000 7f9df6ca7700 1 -- start start 2023-12-11T19:23:18.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.849+0000 7f9df6ca7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df0106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.849+0000 7f9df6ca7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0107510 con 0x7f9df0106b20 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.850+0000 7f9df4a43700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df0106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.850+0000 7f9df4a43700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df0106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33794/0 (socket says 172.21.15.66:33794) 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.850+0000 7f9df4a43700 1 -- 172.21.15.66:0/2410649440 learned_addr learned my addr 172.21.15.66:0/2410649440 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.850+0000 7f9df4a43700 1 -- 172.21.15.66:0/2410649440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0107d20 con 0x7f9df0106b20 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.850+0000 7f9df4a43700 1 --2- 172.21.15.66:0/2410649440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df0106f40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9de40098a0 tx=0x7f9de4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c12e3f8e838e80 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9def7fe700 1 -- 172.21.15.66:0/2410649440 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de4004160 con 0x7f9df0106b20 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9def7fe700 1 -- 172.21.15.66:0/2410649440 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9de40042c0 con 0x7f9df0106b20 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9def7fe700 1 -- 172.21.15.66:0/2410649440 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de4011940 con 0x7f9df0106b20 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2410649440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 msgr2=0x7f9df0106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 --2- 172.21.15.66:0/2410649440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df0106f40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9de40098a0 tx=0x7f9de4009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2410649440 shutdown_connections 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 --2- 172.21.15.66:0/2410649440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df0106f40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2410649440 >> 172.21.15.66:0/2410649440 conn(0x7f9df0102080 msgr2=0x7f9df0104500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2410649440 shutdown_connections 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.851+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2410649440 wait complete. 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.852+0000 7f9df6ca7700 1 Processor -- start 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.853+0000 7f9df6ca7700 1 -- start start 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.853+0000 7f9df6ca7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.853+0000 7f9df6ca7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019cef0 con 0x7f9df0106b20 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.853+0000 7f9df4a43700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.853+0000 7f9df4a43700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33798/0 (socket says 172.21.15.66:33798) 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.853+0000 7f9df4a43700 1 -- 172.21.15.66:0/2742002669 learned_addr learned my addr 172.21.15.66:0/2742002669 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9df4a43700 1 -- 172.21.15.66:0/2742002669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4009550 con 0x7f9df0106b20 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9df4a43700 1 --2- 172.21.15.66:0/2742002669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df019c9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9de4003f20 tx=0x7f9de4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de40036a0 con 0x7f9df0106b20 2023-12-11T19:23:18.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9de401b440 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df019d0f0 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9de4023400 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.854+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df019d590 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.855+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9de40239c0 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.855+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9de40193d0 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.855+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df0062c00 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.858+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9de4023db0 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.936+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9df01a00c0 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.936+0000 7f9dedffb700 1 -- 172.21.15.66:0/2742002669 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9de401b5b0 con 0x7f9df0106b20 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 msgr2=0x7f9df019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 --2- 172.21.15.66:0/2742002669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df019c9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9de4003f20 tx=0x7f9de4011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 shutdown_connections 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 --2- 172.21.15.66:0/2742002669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9df0106b20 0x7f9df019c9b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:18.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 >> 172.21.15.66:0/2742002669 conn(0x7f9df0102080 msgr2=0x7f9df0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:18.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 shutdown_connections 2023-12-11T19:23:18.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:17.937+0000 7f9df6ca7700 1 -- 172.21.15.66:0/2742002669 wait complete. 2023-12-11T19:23:18.376 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (1/15)... 2023-12-11T19:23:18.737 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:18.436+0000 7f6236970200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T19:23:18.737 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:18.614+0000 7f6236970200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T19:23:19.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:18.736+0000 7f6236970200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T19:23:19.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:18.866+0000 7f6236970200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T19:23:19.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:19.445+0000 7f6236970200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T19:23:20.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:20.394+0000 7f6236970200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T19:23:20.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:20.530+0000 7f6236970200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T19:23:21.923 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:21.556+0000 7f6236970200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:23:21.923 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:21.686+0000 7f6236970200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T19:23:22.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:21.922+0000 7f6236970200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:23:22.809 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:22.482+0000 7f6236970200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T19:23:22.809 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:22.604+0000 7f6236970200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T19:23:23.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:22.808+0000 7f6236970200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T19:23:23.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:23.171+0000 7f6236970200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T19:23:24.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:24.161+0000 7f6236970200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T19:23:24.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:24.346+0000 7f6236970200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T19:23:25.058 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:24.609+0000 7f6236970200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T19:23:25.058 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:24.762+0000 7f6236970200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T19:23:25.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:25 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:25.057+0000 7f6236970200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T19:23:26.462 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:26.198+0000 7f6236970200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: Activating manager daemon x 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: mgrmap e2: x(active, starting, since 0.0043499s) 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: Manager daemon x is now available 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T19:23:26.463 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T19:23:26.464 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' 2023-12-11T19:23:26.464 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:26 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' 2023-12-11T19:23:27.185 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-11T19:23:27.188 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-11T19:23:27.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-11T19:23:27.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-11T19:23:27.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-11T19:23:27.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-11T19:23:27.198 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2023-12-11T19:23:09.144924+0000", 2023-12-11T19:23:27.199 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.668+0000 7fa31a783700 1 Processor -- start 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.668+0000 7fa31a783700 1 -- start start 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.668+0000 7fa31a783700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa314109990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.668+0000 7fa31a783700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31407ae50 con 0x7fa3141075a0 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.669+0000 7fa313fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa314109990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.669+0000 7fa313fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa314109990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52026/0 (socket says 172.21.15.66:52026) 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.669+0000 7fa313fff700 1 -- 172.21.15.66:0/1314743491 learned_addr learned my addr 172.21.15.66:0/1314743491 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.669+0000 7fa313fff700 1 -- 172.21.15.66:0/1314743491 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314109ed0 con 0x7fa3141075a0 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.669+0000 7fa313fff700 1 --2- 172.21.15.66:0/1314743491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa314109990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098a0 tx=0x7fa304009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afcd7410f3dcf4e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa312ffd700 1 -- 172.21.15.66:0/1314743491 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa304004160 con 0x7fa3141075a0 2023-12-11T19:23:27.200 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa312ffd700 1 -- 172.21.15.66:0/1314743491 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa3040042c0 con 0x7fa3141075a0 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa312ffd700 1 -- 172.21.15.66:0/1314743491 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa304011940 con 0x7fa3141075a0 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa31a783700 1 -- 172.21.15.66:0/1314743491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 msgr2=0x7fa314109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa31a783700 1 --2- 172.21.15.66:0/1314743491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa314109990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098a0 tx=0x7fa304009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa31a783700 1 -- 172.21.15.66:0/1314743491 shutdown_connections 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa31a783700 1 --2- 172.21.15.66:0/1314743491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa314109990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa31a783700 1 -- 172.21.15.66:0/1314743491 >> 172.21.15.66:0/1314743491 conn(0x7fa314101010 msgr2=0x7fa314103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.670+0000 7fa31a783700 1 -- 172.21.15.66:0/1314743491 shutdown_connections 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.671+0000 7fa31a783700 1 -- 172.21.15.66:0/1314743491 wait complete. 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.671+0000 7fa31a783700 1 Processor -- start 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.672+0000 7fa31a783700 1 -- start start 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.672+0000 7fa31a783700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa31419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.672+0000 7fa31a783700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31419ac90 con 0x7fa3141075a0 2023-12-11T19:23:27.201 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.672+0000 7fa313fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa31419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.672+0000 7fa313fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa31419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52040/0 (socket says 172.21.15.66:52040) 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.672+0000 7fa313fff700 1 -- 172.21.15.66:0/2744545940 learned_addr learned my addr 172.21.15.66:0/2744545940 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa313fff700 1 -- 172.21.15.66:0/2744545940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009550 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa313fff700 1 --2- 172.21.15.66:0/2744545940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa31419a750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa304003f20 tx=0x7fa304011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3040037d0 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa30401b440 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31419ae90 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa304023590 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.673+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31419b330 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.674+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 2) v1 ==== 45336+0+0 (secure 0 0 0) 0x7fa30401b5b0 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.674+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31410ac10 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.675+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa30404c3d0 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.677+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa30401b920 con 0x7fa3141075a0 2023-12-11T19:23:27.202 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.755+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa31410ae50 con 0x7fa3141075a0 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.756+0000 7fa3117fa700 1 -- 172.21.15.66:0/2744545940 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fa304016260 con 0x7fa3141075a0 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.756+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 msgr2=0x7fa31419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.756+0000 7fa31a783700 1 --2- 172.21.15.66:0/2744545940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa31419a750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa304003f20 tx=0x7fa304011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.757+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 shutdown_connections 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.757+0000 7fa31a783700 1 --2- 172.21.15.66:0/2744545940 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3141075a0 0x7fa31419a750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.757+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 >> 172.21.15.66:0/2744545940 conn(0x7fa314101010 msgr2=0x7fa314101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.757+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 shutdown_connections 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:26.757+0000 7fa31a783700 1 -- 172.21.15.66:0/2744545940 wait complete. 2023-12-11T19:23:27.203 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (2/15)... 2023-12-11T19:23:27.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:27 smithi066 ceph-mon[97920]: from='mgr.14100 172.21.15.66:0/3437740057' entity='mgr.x' 2023-12-11T19:23:27.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:27 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2744545940' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-11T19:23:27.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:27 smithi066 ceph-mon[97920]: mgrmap e3: x(active, since 1.00838s) 2023-12-11T19:23:29.525 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:29 smithi066 ceph-mon[97920]: mgrmap e4: x(active, since 2s) 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-11T19:23:30.487 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-11T19:23:30.488 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-11T19:23:30.489 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-11T19:23:30.498 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-11T19:23:30.499 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2023-12-11T19:23:09.144924+0000", 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.734+0000 7f80e1a42700 1 Processor -- start 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.734+0000 7f80e1a42700 1 -- start start 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.735+0000 7f80e1a42700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:30.500 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.735+0000 7f80e1a42700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc107510 con 0x7f80dc106b20 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.735+0000 7f80daffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.735+0000 7f80daffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52046/0 (socket says 172.21.15.66:52046) 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.735+0000 7f80daffd700 1 -- 172.21.15.66:0/1245314 learned_addr learned my addr 172.21.15.66:0/1245314 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.735+0000 7f80daffd700 1 -- 172.21.15.66:0/1245314 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc107d20 con 0x7f80dc106b20 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.736+0000 7f80daffd700 1 --2- 172.21.15.66:0/1245314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc106f40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0098a0 tx=0x7f80cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7931b6bd79763f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.736+0000 7f80d9ffb700 1 -- 172.21.15.66:0/1245314 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80cc004160 con 0x7f80dc106b20 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.736+0000 7f80d9ffb700 1 -- 172.21.15.66:0/1245314 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f80cc0042c0 con 0x7f80dc106b20 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.736+0000 7f80d9ffb700 1 -- 172.21.15.66:0/1245314 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80cc011940 con 0x7f80dc106b20 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 -- 172.21.15.66:0/1245314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 msgr2=0x7f80dc106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 --2- 172.21.15.66:0/1245314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc106f40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0098a0 tx=0x7f80cc009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 -- 172.21.15.66:0/1245314 shutdown_connections 2023-12-11T19:23:30.501 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 --2- 172.21.15.66:0/1245314 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc106f40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 -- 172.21.15.66:0/1245314 >> 172.21.15.66:0/1245314 conn(0x7f80dc102080 msgr2=0x7f80dc104500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 -- 172.21.15.66:0/1245314 shutdown_connections 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.737+0000 7f80e1a42700 1 -- 172.21.15.66:0/1245314 wait complete. 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.738+0000 7f80e1a42700 1 Processor -- start 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.738+0000 7f80e1a42700 1 -- start start 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.738+0000 7f80e1a42700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.738+0000 7f80e1a42700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc19cf40 con 0x7f80dc106b20 2023-12-11T19:23:30.502 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.739+0000 7f80daffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.739+0000 7f80daffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52060/0 (socket says 172.21.15.66:52060) 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.739+0000 7f80daffd700 1 -- 172.21.15.66:0/3660569634 learned_addr learned my addr 172.21.15.66:0/3660569634 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.739+0000 7f80daffd700 1 -- 172.21.15.66:0/3660569634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc009550 con 0x7f80dc106b20 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.739+0000 7f80daffd700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc19ca00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f80cc003f20 tx=0x7f80cc011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.739+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80cc0036a0 con 0x7f80dc106b20 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.740+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f80cc01b440 con 0x7f80dc106b20 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.740+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80cc023400 con 0x7f80dc106b20 2023-12-11T19:23:30.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.740+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc19d140 con 0x7f80dc106b20 2023-12-11T19:23:30.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.740+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc19d5e0 con 0x7f80dc106b20 2023-12-11T19:23:30.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.741+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f80cc01b5b0 con 0x7f80dc106b20 2023-12-11T19:23:30.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.741+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc10af50 con 0x7f80dc106b20 2023-12-11T19:23:30.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.741+0000 7f80e0a40700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7f80c40387d0 0x7f80c403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:30.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.741+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f80cc04c7d0 con 0x7f80dc106b20 2023-12-11T19:23:30.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.742+0000 7f80da7fc700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7f80c40387d0 0x7f80c403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.742+0000 7f80da7fc700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7f80c40387d0 0x7f80c403ac90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80d0006fd0 tx=0x7f80d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:29.747+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f80cc00bc10 con 0x7f80dc106b20 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.039+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f80dc19fe70 con 0x7f80dc106b20 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.040+0000 7f80e0a40700 1 -- 172.21.15.66:0/3660569634 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f80cc00b3e0 con 0x7f80dc106b20 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7f80c40387d0 msgr2=0x7f80c403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7f80c40387d0 0x7f80c403ac90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80d0006fd0 tx=0x7f80d0006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:30.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 msgr2=0x7f80dc19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc19ca00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f80cc003f20 tx=0x7f80cc011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 shutdown_connections 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7f80c40387d0 0x7f80c403ac90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 --2- 172.21.15.66:0/3660569634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80dc106b20 0x7f80dc19ca00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.042+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 >> 172.21.15.66:0/3660569634 conn(0x7f80dc102080 msgr2=0x7f80dc102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.043+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 shutdown_connections 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:30.043+0000 7f80e1a42700 1 -- 172.21.15.66:0/3660569634 wait complete. 2023-12-11T19:23:30.506 INFO:teuthology.orchestra.run.smithi066.stdout:mgr is available 2023-12-11T19:23:31.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:30 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3660569634' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-11T19:23:31.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.044+0000 7ff0637f9700 1 Processor -- start 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.044+0000 7ff0637f9700 1 -- start start 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.044+0000 7ff0637f9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c1099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:31.682 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.044+0000 7ff0637f9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c07adf0 con 0x7ff05c107600 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.045+0000 7ff061595700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c1099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.045+0000 7ff061595700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c1099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52064/0 (socket says 172.21.15.66:52064) 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.045+0000 7ff061595700 1 -- 172.21.15.66:0/3381548092 learned_addr learned my addr 172.21.15.66:0/3381548092 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.045+0000 7ff061595700 1 -- 172.21.15.66:0/3381548092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c07af30 con 0x7ff05c107600 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.045+0000 7ff061595700 1 --2- 172.21.15.66:0/3381548092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c1099f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0580098a0 tx=0x7ff058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996455492ea7aca2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.045+0000 7ff053fff700 1 -- 172.21.15.66:0/3381548092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff058004160 con 0x7ff05c107600 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff053fff700 1 -- 172.21.15.66:0/3381548092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff0580042c0 con 0x7ff05c107600 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff053fff700 1 -- 172.21.15.66:0/3381548092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff058011940 con 0x7ff05c107600 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 -- 172.21.15.66:0/3381548092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 msgr2=0x7ff05c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 --2- 172.21.15.66:0/3381548092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c1099f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0580098a0 tx=0x7ff058009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 -- 172.21.15.66:0/3381548092 shutdown_connections 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 --2- 172.21.15.66:0/3381548092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c1099f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 -- 172.21.15.66:0/3381548092 >> 172.21.15.66:0/3381548092 conn(0x7ff05c101010 msgr2=0x7ff05c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:31.683 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 -- 172.21.15.66:0/3381548092 shutdown_connections 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.046+0000 7ff0637f9700 1 -- 172.21.15.66:0/3381548092 wait complete. 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.047+0000 7ff0637f9700 1 Processor -- start 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff0637f9700 1 -- start start 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff0637f9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff0637f9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c107370 con 0x7ff05c107600 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff061595700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff061595700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52076/0 (socket says 172.21.15.66:52076) 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff061595700 1 -- 172.21.15.66:0/2556603067 learned_addr learned my addr 172.21.15.66:0/2556603067 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.048+0000 7ff061595700 1 -- 172.21.15.66:0/2556603067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058009550 con 0x7ff05c107600 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.049+0000 7ff061595700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c106e30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff058003f20 tx=0x7ff058011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.049+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0580037d0 con 0x7ff05c107600 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.049+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff05801b440 con 0x7ff05c107600 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.049+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff058023590 con 0x7ff05c107600 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.049+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c1055b0 con 0x7ff05c107600 2023-12-11T19:23:31.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.049+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c105a50 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.050+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7ff05801b5b0 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.050+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c062c00 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.051+0000 7ff0527fc700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff048038820 0x7ff04803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.051+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff05804c770 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.051+0000 7ff060d94700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff048038820 0x7ff04803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.052+0000 7ff060d94700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff048038820 0x7ff04803ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff04c006fd0 tx=0x7ff04c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.056+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff05801b920 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.264+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff05c105e00 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.265+0000 7ff0527fc700 1 -- 172.21.15.66:0/2556603067 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff058016260 con 0x7ff05c107600 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.267+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff048038820 msgr2=0x7ff04803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.267+0000 7ff0637f9700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff048038820 0x7ff04803ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff04c006fd0 tx=0x7ff04c006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.267+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 msgr2=0x7ff05c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:31.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.267+0000 7ff0637f9700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c106e30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff058003f20 tx=0x7ff058011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.268+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 shutdown_connections 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.268+0000 7ff0637f9700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff048038820 0x7ff04803ace0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.268+0000 7ff0637f9700 1 --2- 172.21.15.66:0/2556603067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff05c107600 0x7ff05c106e30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.268+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 >> 172.21.15.66:0/2556603067 conn(0x7ff05c101010 msgr2=0x7ff05c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.268+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 shutdown_connections 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:31.268+0000 7ff0637f9700 1 -- 172.21.15.66:0/2556603067 wait complete. 2023-12-11T19:23:31.686 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling cephadm module... 2023-12-11T19:23:31.973 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:31 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2556603067' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-11T19:23:32.974 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:32 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4074776481' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-11T19:23:32.974 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: ignoring --setuser ceph since I am not root 2023-12-11T19:23:32.974 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: ignoring --setgroup ceph since I am not root 2023-12-11T19:23:33.164 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.203+0000 7fa831b30700 1 Processor -- start 2023-12-11T19:23:33.164 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.203+0000 7fa831b30700 1 -- start start 2023-12-11T19:23:33.164 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.203+0000 7fa831b30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:33.164 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.203+0000 7fa831b30700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c07b890 con 0x7fa82c07ce60 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.204+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.204+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c07b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40062/0 (socket says 172.21.15.66:40062) 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.204+0000 7fa82b7fe700 1 -- 172.21.15.66:0/3857217294 learned_addr learned my addr 172.21.15.66:0/3857217294 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.204+0000 7fa82b7fe700 1 -- 172.21.15.66:0/3857217294 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c07b9d0 con 0x7fa82c07ce60 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.204+0000 7fa82b7fe700 1 --2- 172.21.15.66:0/3857217294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c07b2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0098a0 tx=0x7fa81c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63952cc61f30aa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.204+0000 7fa82a7fc700 1 -- 172.21.15.66:0/3857217294 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa81c004160 con 0x7fa82c07ce60 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa82a7fc700 1 -- 172.21.15.66:0/3857217294 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa81c0042c0 con 0x7fa82c07ce60 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa82a7fc700 1 -- 172.21.15.66:0/3857217294 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa81c011940 con 0x7fa82c07ce60 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 -- 172.21.15.66:0/3857217294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 msgr2=0x7fa82c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 --2- 172.21.15.66:0/3857217294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c07b2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0098a0 tx=0x7fa81c009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 -- 172.21.15.66:0/3857217294 shutdown_connections 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 --2- 172.21.15.66:0/3857217294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c07b2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:33.165 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 -- 172.21.15.66:0/3857217294 >> 172.21.15.66:0/3857217294 conn(0x7fa82c1020c0 msgr2=0x7fa82c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 -- 172.21.15.66:0/3857217294 shutdown_connections 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.205+0000 7fa831b30700 1 -- 172.21.15.66:0/3857217294 wait complete. 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.206+0000 7fa831b30700 1 Processor -- start 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa831b30700 1 -- start start 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa831b30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa831b30700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c19cea0 con 0x7fa82c07ce60 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40066/0 (socket says 172.21.15.66:40066) 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa82b7fe700 1 -- 172.21.15.66:0/4074776481 learned_addr learned my addr 172.21.15.66:0/4074776481 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.207+0000 7fa82b7fe700 1 -- 172.21.15.66:0/4074776481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c009550 con 0x7fa82c07ce60 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.208+0000 7fa82b7fe700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c19c960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa81c003f20 tx=0x7fa81c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.208+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa81c0037d0 con 0x7fa82c07ce60 2023-12-11T19:23:33.166 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.208+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa81c01b440 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.208+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c19d0a0 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.208+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa81c023590 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.208+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c19d540 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.209+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fa81c003930 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.209+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c10ce90 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.210+0000 7fa828ff9700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa814038820 0x7fa81403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.210+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa81c04c770 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.210+0000 7fa82affd700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa814038820 0x7fa81403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.210+0000 7fa82affd700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa814038820 0x7fa81403ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.215+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa81c00bbd0 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.483+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa82c10d0d0 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.727+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa81c00bbd0 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.728+0000 7fa828ff9700 1 -- 172.21.15.66:0/4074776481 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fa81c00b3e0 con 0x7fa82c07ce60 2023-12-11T19:23:33.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa814038820 msgr2=0x7fa81403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa814038820 0x7fa81403ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 msgr2=0x7fa82c19c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c19c960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa81c003f20 tx=0x7fa81c011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 shutdown_connections 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa814038820 0x7fa81403ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 --2- 172.21.15.66:0/4074776481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa82c07ce60 0x7fa82c19c960 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.730+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 >> 172.21.15.66:0/4074776481 conn(0x7fa82c1020c0 msgr2=0x7fa82c102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.731+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 shutdown_connections 2023-12-11T19:23:33.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:32.731+0000 7fa831b30700 1 -- 172.21.15.66:0/4074776481 wait complete. 2023-12-11T19:23:33.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:33.001+0000 7f5956a42200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T19:23:33.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:33.112+0000 7f5956a42200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T19:23:33.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:33.240+0000 7f5956a42200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T19:23:33.629 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:33.368+0000 7f5956a42200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T19:23:34.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4074776481' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-11T19:23:34.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-mon[97920]: mgrmap e5: x(active, since 6s) 2023-12-11T19:23:34.039 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:33.627+0000 7f5956a42200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T19:23:34.310 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:34.135+0000 7f5956a42200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.728+0000 7fa72d58d700 1 Processor -- start 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.728+0000 7fa72d58d700 1 -- start start 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.729+0000 7fa72d58d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa728109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:34.459 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.729+0000 7fa72d58d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728109740 con 0x7fa728108d50 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.729+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa728109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.729+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa728109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40084/0 (socket says 172.21.15.66:40084) 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.729+0000 7fa726ffd700 1 -- 172.21.15.66:0/3799966462 learned_addr learned my addr 172.21.15.66:0/3799966462 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.729+0000 7fa726ffd700 1 -- 172.21.15.66:0/3799966462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728109f50 con 0x7fa728108d50 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.730+0000 7fa726ffd700 1 --2- 172.21.15.66:0/3799966462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa728109170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098a0 tx=0x7fa718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d89d0770f83c8ff server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.730+0000 7fa725ffb700 1 -- 172.21.15.66:0/3799966462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa718004160 con 0x7fa728108d50 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.730+0000 7fa725ffb700 1 -- 172.21.15.66:0/3799966462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa7180042c0 con 0x7fa728108d50 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.730+0000 7fa725ffb700 1 -- 172.21.15.66:0/3799966462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa718011940 con 0x7fa728108d50 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 -- 172.21.15.66:0/3799966462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 msgr2=0x7fa728109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 --2- 172.21.15.66:0/3799966462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa728109170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098a0 tx=0x7fa718009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 -- 172.21.15.66:0/3799966462 shutdown_connections 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 --2- 172.21.15.66:0/3799966462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa728109170 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 -- 172.21.15.66:0/3799966462 >> 172.21.15.66:0/3799966462 conn(0x7fa72807c0e0 msgr2=0x7fa72807c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:34.460 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 -- 172.21.15.66:0/3799966462 shutdown_connections 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.731+0000 7fa72d58d700 1 -- 172.21.15.66:0/3799966462 wait complete. 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.732+0000 7fa72d58d700 1 Processor -- start 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.732+0000 7fa72d58d700 1 -- start start 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.732+0000 7fa72d58d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa72819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.732+0000 7fa72d58d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819cef0 con 0x7fa728108d50 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.733+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa72819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.733+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa72819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40088/0 (socket says 172.21.15.66:40088) 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.733+0000 7fa726ffd700 1 -- 172.21.15.66:0/1107566099 learned_addr learned my addr 172.21.15.66:0/1107566099 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.733+0000 7fa726ffd700 1 -- 172.21.15.66:0/1107566099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718009550 con 0x7fa728108d50 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.733+0000 7fa726ffd700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa72819c9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa718003f20 tx=0x7fa718011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.734+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7180036a0 con 0x7fa728108d50 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.734+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa71801b440 con 0x7fa728108d50 2023-12-11T19:23:34.461 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.734+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa718023400 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.734+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72819d0f0 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.734+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72819d590 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.735+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72810d180 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.735+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fa71801b5b0 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.736+0000 7fa70ffff700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 0x7fa71003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.736+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa71804c7d0 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.736+0000 7fa7267fc700 1 -- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 msgr2=0x7fa71003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.736+0000 7fa7267fc700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 0x7fa71003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.741+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa71801b920 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.937+0000 7fa7267fc700 1 -- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 msgr2=0x7fa71003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:33.937+0000 7fa7267fc700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 0x7fa71003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.012+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa72819fe80 con 0x7fa728108d50 2023-12-11T19:23:34.462 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.013+0000 7fa70ffff700 1 -- 172.21.15.66:0/1107566099 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fa71800b3e0 con 0x7fa728108d50 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 msgr2=0x7fa71003ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 0x7fa71003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 msgr2=0x7fa72819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa72819c9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa718003f20 tx=0x7fa718011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 shutdown_connections 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7fa710038870 0x7fa71003ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 --2- 172.21.15.66:0/1107566099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa728108d50 0x7fa72819c9b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 >> 172.21.15.66:0/1107566099 conn(0x7fa72807c0e0 msgr2=0x7fa728106410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 shutdown_connections 2023-12-11T19:23:34.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.015+0000 7fa72d58d700 1 -- 172.21.15.66:0/1107566099 wait complete. 2023-12-11T19:23:34.464 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2023-12-11T19:23:34.464 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 5... 2023-12-11T19:23:34.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:34.308+0000 7f5956a42200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T19:23:34.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:34.429+0000 7f5956a42200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T19:23:35.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:34 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1107566099' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-11T19:23:35.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:34.566+0000 7f5956a42200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T19:23:35.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:35.152+0000 7f5956a42200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T19:23:36.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:36.049+0000 7f5956a42200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T19:23:36.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:36.160+0000 7f5956a42200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T19:23:37.529 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:37.166+0000 7f5956a42200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:23:37.530 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:37.292+0000 7f5956a42200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T19:23:37.530 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:37.527+0000 7f5956a42200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:23:38.403 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:38.076+0000 7f5956a42200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T19:23:38.403 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:38.198+0000 7f5956a42200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T19:23:38.767 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:38.401+0000 7f5956a42200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T19:23:39.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:38.765+0000 7f5956a42200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T19:23:40.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:39.741+0000 7f5956a42200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T19:23:40.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:39.925+0000 7f5956a42200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T19:23:40.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:40.188+0000 7f5956a42200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T19:23:40.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:40.340+0000 7f5956a42200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T19:23:41.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:40.631+0000 7f5956a42200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:23:41.753+0000 7f5956a42200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: Active manager daemon x restarted 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: Activating manager daemon x 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: osdmap e2: 0 total, 0 up, 0 in 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: mgrmap e6: x(active, starting, since 0.00386684s) 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T19:23:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T19:23:42.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T19:23:42.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T19:23:42.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: Manager daemon x is now available 2023-12-11T19:23:42.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:42.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:41 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: Found migration_current of "None". Setting to last migration. 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:43.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:42 smithi066 ceph-mon[97920]: mgrmap e7: x(active, since 1.00688s) 2023-12-11T19:23:43.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:23:43.167 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.994+0000 7ff4817f8700 1 Processor -- start 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.995+0000 7ff4817f8700 1 -- start start 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.995+0000 7ff4817f8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.995+0000 7ff4817f8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c07b8b0 con 0x7ff47c07ce80 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.995+0000 7ff47affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.995+0000 7ff47affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c07b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40098/0 (socket says 172.21.15.66:40098) 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.995+0000 7ff47affd700 1 -- 172.21.15.66:0/2369694402 learned_addr learned my addr 172.21.15.66:0/2369694402 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.996+0000 7ff47affd700 1 -- 172.21.15.66:0/2369694402 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c07b9f0 con 0x7ff47c07ce80 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.996+0000 7ff47affd700 1 --2- 172.21.15.66:0/2369694402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c07b2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0098a0 tx=0x7ff46c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cf91f06fb1ffe9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.996+0000 7ff479ffb700 1 -- 172.21.15.66:0/2369694402 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff46c004160 con 0x7ff47c07ce80 2023-12-11T19:23:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.996+0000 7ff479ffb700 1 -- 172.21.15.66:0/2369694402 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff46c0042c0 con 0x7ff47c07ce80 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.996+0000 7ff479ffb700 1 -- 172.21.15.66:0/2369694402 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff46c011940 con 0x7ff47c07ce80 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 -- 172.21.15.66:0/2369694402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 msgr2=0x7ff47c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 --2- 172.21.15.66:0/2369694402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c07b2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0098a0 tx=0x7ff46c009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 -- 172.21.15.66:0/2369694402 shutdown_connections 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 --2- 172.21.15.66:0/2369694402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c07b2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 -- 172.21.15.66:0/2369694402 >> 172.21.15.66:0/2369694402 conn(0x7ff47c1020c0 msgr2=0x7ff47c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 -- 172.21.15.66:0/2369694402 shutdown_connections 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.997+0000 7ff4817f8700 1 -- 172.21.15.66:0/2369694402 wait complete. 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.998+0000 7ff4817f8700 1 Processor -- start 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.998+0000 7ff4817f8700 1 -- start start 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.999+0000 7ff4817f8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.999+0000 7ff4817f8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c19cf20 con 0x7ff47c07ce80 2023-12-11T19:23:43.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.999+0000 7ff47affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.999+0000 7ff47affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40100/0 (socket says 172.21.15.66:40100) 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.999+0000 7ff47affd700 1 -- 172.21.15.66:0/3946515181 learned_addr learned my addr 172.21.15.66:0/3946515181 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:34.999+0000 7ff47affd700 1 -- 172.21.15.66:0/3946515181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c009550 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.000+0000 7ff47affd700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c19c9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff46c003f20 tx=0x7ff46c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.000+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff46c0036a0 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.000+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff46c01b440 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.000+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff46c023400 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.000+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c19d120 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.000+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c19d5c0 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.001+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7ff46c01b5b0 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.002+0000 7ff463fff700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.002+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 --> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff46403b440 con 0x7ff464038870 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.002+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff46c04c7d0 con 0x7ff47c07ce80 2023-12-11T19:23:43.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.002+0000 7ff47a7fc700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.002+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.203+0000 7ff47a7fc700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.203+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.604+0000 7ff47a7fc700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:35.604+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:36.405+0000 7ff47a7fc700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:36.405+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:38.007+0000 7ff47a7fc700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:38.007+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:41.210+0000 7ff47a7fc700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3525772822 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:41.210+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:41.758+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mgrmap(e 6) v1 ==== 45347+0+0 (secure 0 0 0) 0x7ff46c003800 con 0x7ff47c07ce80 2023-12-11T19:23:43.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:41.758+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 msgr2=0x7ff46403ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:41.758+0000 7ff463fff700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.759+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7ff46c04e2e0 con 0x7ff47c07ce80 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.759+0000 7ff463fff700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7ff46403bfd0 0x7ff46403e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.759+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff46403b440 con 0x7ff46403bfd0 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.760+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7ff46403bfd0 0x7ff46403e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.761+0000 7ff47a7fc700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7ff46403bfd0 0x7ff46403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff470003a10 tx=0x7ff4700092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.762+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff46403b440 con 0x7ff46403bfd0 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.769+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff47c10ce70 con 0x7ff46403bfd0 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.769+0000 7ff463fff700 1 -- 172.21.15.66:0/3946515181 <== mgr.14118 v2:172.21.15.66:6800/4268281106 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff47c10ce70 con 0x7ff46403bfd0 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7ff46403bfd0 msgr2=0x7ff46403e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7ff46403bfd0 0x7ff46403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff470003a10 tx=0x7ff4700092b0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 msgr2=0x7ff47c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c19c9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff46c003f20 tx=0x7ff46c011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:43.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 shutdown_connections 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7ff46403bfd0 0x7ff46403e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff47c07ce80 0x7ff47c19c9e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 --2- 172.21.15.66:0/3946515181 >> [v2:172.21.15.66:6800/3525772822,v1:172.21.15.66:6801/3525772822] conn(0x7ff464038870 0x7ff46403ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 >> 172.21.15.66:0/3946515181 conn(0x7ff47c1020c0 msgr2=0x7ff47c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 shutdown_connections 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:42.770+0000 7ff4817f8700 1 -- 172.21.15.66:0/3946515181 wait complete. 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 5 is available 2023-12-11T19:23:43.173 INFO:teuthology.orchestra.run.smithi066.stdout:Setting orchestrator backend to cephadm... 2023-12-11T19:23:44.080 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:43 smithi066 ceph-mon[97920]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-11T19:23:44.080 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:43 smithi066 ceph-mon[97920]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-11T19:23:44.080 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:43 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:23:44.355 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.676+0000 7fed5f693700 1 Processor -- start 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.677+0000 7fed5f693700 1 -- start start 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.677+0000 7fed5f693700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5807b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.677+0000 7fed5f693700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5807b870 con 0x7fed5807ce40 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.677+0000 7fed5d42f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5807b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.677+0000 7fed5d42f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5807b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40274/0 (socket says 172.21.15.66:40274) 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.677+0000 7fed5d42f700 1 -- 172.21.15.66:0/2433521008 learned_addr learned my addr 172.21.15.66:0/2433521008 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.678+0000 7fed5d42f700 1 -- 172.21.15.66:0/2433521008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5807b9b0 con 0x7fed5807ce40 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.678+0000 7fed5d42f700 1 --2- 172.21.15.66:0/2433521008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5807b2a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed540098a0 tx=0x7fed54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b965325c606f02 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.678+0000 7fed4ffff700 1 -- 172.21.15.66:0/2433521008 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed54004160 con 0x7fed5807ce40 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.678+0000 7fed4ffff700 1 -- 172.21.15.66:0/2433521008 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fed540042c0 con 0x7fed5807ce40 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.678+0000 7fed4ffff700 1 -- 172.21.15.66:0/2433521008 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed54011940 con 0x7fed5807ce40 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 -- 172.21.15.66:0/2433521008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 msgr2=0x7fed5807b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:44.356 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 --2- 172.21.15.66:0/2433521008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5807b2a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed540098a0 tx=0x7fed54009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 -- 172.21.15.66:0/2433521008 shutdown_connections 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 --2- 172.21.15.66:0/2433521008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5807b2a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 -- 172.21.15.66:0/2433521008 >> 172.21.15.66:0/2433521008 conn(0x7fed581020c0 msgr2=0x7fed58104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 -- 172.21.15.66:0/2433521008 shutdown_connections 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.679+0000 7fed5f693700 1 -- 172.21.15.66:0/2433521008 wait complete. 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.680+0000 7fed5f693700 1 Processor -- start 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.680+0000 7fed5f693700 1 -- start start 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5f693700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5f693700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5819ceb0 con 0x7fed5807ce40 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5d42f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5d42f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40282/0 (socket says 172.21.15.66:40282) 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5d42f700 1 -- 172.21.15.66:0/2209727991 learned_addr learned my addr 172.21.15.66:0/2209727991 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5d42f700 1 -- 172.21.15.66:0/2209727991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54009550 con 0x7fed5807ce40 2023-12-11T19:23:44.357 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.681+0000 7fed5d42f700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5819c970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fed54003f20 tx=0x7fed54011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.682+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed540037d0 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.682+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fed5401b440 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.682+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed54023590 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.682+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5819d0b0 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.682+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5819d550 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.683+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7fed5401b5b0 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.683+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5810ce50 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.683+0000 7fed4e7fc700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fed44038740 0x7fed4403ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.683+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed5404c6a0 con 0x7fed5807ce40 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.684+0000 7fed5cc2e700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fed44038740 0x7fed4403ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:44.358 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.684+0000 7fed5cc2e700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fed44038740 0x7fed4403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed48006fd0 tx=0x7fed48006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.689+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fed5401b920 con 0x7fed5807ce40 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.810+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fed54003930 con 0x7fed5807ce40 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.920+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fed58103370 con 0x7fed44038740 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.927+0000 7fed4e7fc700 1 -- 172.21.15.66:0/2209727991 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fed58103370 con 0x7fed44038740 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.929+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fed44038740 msgr2=0x7fed4403ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.929+0000 7fed5f693700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fed44038740 0x7fed4403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed48006fd0 tx=0x7fed48006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.929+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 msgr2=0x7fed5819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.929+0000 7fed5f693700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5819c970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fed54003f20 tx=0x7fed54011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.930+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 shutdown_connections 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.930+0000 7fed5f693700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fed44038740 0x7fed4403ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.930+0000 7fed5f693700 1 --2- 172.21.15.66:0/2209727991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed5807ce40 0x7fed5819c970 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:44.359 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.930+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 >> 172.21.15.66:0/2209727991 conn(0x7fed581020c0 msgr2=0x7fed58102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:44.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.930+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 shutdown_connections 2023-12-11T19:23:44.360 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:43.930+0000 7fed5f693700 1 -- 172.21.15.66:0/2209727991 wait complete. 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: [11/Dec/2023:19:23:43] ENGINE Bus STARTING 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: [11/Dec/2023:19:23:43] ENGINE Serving on https://172.21.15.66:7150 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: [11/Dec/2023:19:23:43] ENGINE Serving on http://172.21.15.66:8765 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: [11/Dec/2023:19:23:43] ENGINE Bus STARTED 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: mgrmap e8: x(active, since 2s) 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:45.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:44 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.902+0000 7faac9969700 1 Processor -- start 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.902+0000 7faac9969700 1 -- start start 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.902+0000 7faac9969700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac4106f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.902+0000 7faac9969700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4107550 con 0x7faac4106b60 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.903+0000 7faac2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac4106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.903+0000 7faac2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac4106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40296/0 (socket says 172.21.15.66:40296) 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.903+0000 7faac2ffd700 1 -- 172.21.15.66:0/1544240625 learned_addr learned my addr 172.21.15.66:0/1544240625 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:45.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.903+0000 7faac2ffd700 1 -- 172.21.15.66:0/1544240625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac4107d60 con 0x7faac4106b60 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.903+0000 7faac2ffd700 1 --2- 172.21.15.66:0/1544240625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac4106f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faab40098a0 tx=0x7faab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d05e69448dbb99 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.903+0000 7faac1ffb700 1 -- 172.21.15.66:0/1544240625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faab4004160 con 0x7faac4106b60 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac1ffb700 1 -- 172.21.15.66:0/1544240625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7faab40042c0 con 0x7faac4106b60 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac1ffb700 1 -- 172.21.15.66:0/1544240625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faab4011940 con 0x7faac4106b60 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 -- 172.21.15.66:0/1544240625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 msgr2=0x7faac4106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 --2- 172.21.15.66:0/1544240625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac4106f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faab40098a0 tx=0x7faab4009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 -- 172.21.15.66:0/1544240625 shutdown_connections 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 --2- 172.21.15.66:0/1544240625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac4106f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 -- 172.21.15.66:0/1544240625 >> 172.21.15.66:0/1544240625 conn(0x7faac41020c0 msgr2=0x7faac4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 -- 172.21.15.66:0/1544240625 shutdown_connections 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.904+0000 7faac9969700 1 -- 172.21.15.66:0/1544240625 wait complete. 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.905+0000 7faac9969700 1 Processor -- start 2023-12-11T19:23:45.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac9969700 1 -- start start 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac9969700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac9969700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419cee0 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac419c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40312/0 (socket says 172.21.15.66:40312) 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac2ffd700 1 -- 172.21.15.66:0/1429589851 learned_addr learned my addr 172.21.15.66:0/1429589851 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.906+0000 7faac2ffd700 1 -- 172.21.15.66:0/1429589851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab4009550 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.907+0000 7faac2ffd700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac419c9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faab4003f20 tx=0x7faab4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.907+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faab40036a0 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.907+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7faab401b440 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.907+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faab4023400 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.907+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac419d0e0 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.907+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac419d580 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.908+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7faab401b5b0 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.908+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac410af90 con 0x7faac4106b60 2023-12-11T19:23:45.543 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.909+0000 7faac8967700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7faaac038870 0x7faaac03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.909+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faab404c7e0 con 0x7faac4106b60 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.909+0000 7faac27fc700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7faaac038870 0x7faaac03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.910+0000 7faac27fc700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7faaac038870 0x7faaac03ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faab8006fd0 tx=0x7faab8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:44.914+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faab401b920 con 0x7faac4106b60 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.127+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7faac419fe10 con 0x7faaac038870 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.128+0000 7faac8967700 1 -- 172.21.15.66:0/1429589851 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7faac419fe10 con 0x7faaac038870 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7faaac038870 msgr2=0x7faaac03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7faaac038870 0x7faaac03ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faab8006fd0 tx=0x7faab8006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 msgr2=0x7faac419c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac419c9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faab4003f20 tx=0x7faab4011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 shutdown_connections 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7faaac038870 0x7faaac03ad30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 --2- 172.21.15.66:0/1429589851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faac4106b60 0x7faac419c9a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:45.544 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 >> 172.21.15.66:0/1429589851 conn(0x7faac41020c0 msgr2=0x7faac4102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:45.545 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 shutdown_connections 2023-12-11T19:23:45.545 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:45.130+0000 7faac9969700 1 -- 172.21.15.66:0/1429589851 wait complete. 2023-12-11T19:23:45.545 INFO:teuthology.orchestra.run.smithi066.stdout:Generating ssh key... 2023-12-11T19:23:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:45 smithi066 ceph-mon[97920]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:46.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: Generating public/private rsa key pair. 2023-12-11T19:23:46.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: Your identification has been saved in /tmp/tmpla1lt431/key. 2023-12-11T19:23:46.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: Your public key has been saved in /tmp/tmpla1lt431/key.pub. 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: The key fingerprint is: 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: SHA256:RJwf5VmtWxcTcrXPvl9nPrY+JvSlZdW37JK55/3HcM4 ceph-976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: The key's randomart image is: 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: +---[RSA 3072]----+ 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | ... ....++| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | .o .. oooo| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | .. .o o+| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | . . ..B| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | S .oB| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | .o+=| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | . =X*| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | =.@E| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: | .X=%| 2023-12-11T19:23:46.814 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: +----[SHA256]-----+ 2023-12-11T19:23:46.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.088+0000 7effac839700 1 Processor -- start 2023-12-11T19:23:46.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.089+0000 7effac839700 1 -- start start 2023-12-11T19:23:46.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.089+0000 7effac839700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa81099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:46.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.089+0000 7effac839700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa807ae30 con 0x7effa81075c0 2023-12-11T19:23:46.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.089+0000 7effa659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa81099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:46.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.089+0000 7effa659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa81099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40318/0 (socket says 172.21.15.66:40318) 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.089+0000 7effa659c700 1 -- 172.21.15.66:0/1919223729 learned_addr learned my addr 172.21.15.66:0/1919223729 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.090+0000 7effa659c700 1 -- 172.21.15.66:0/1919223729 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa8109ef0 con 0x7effa81075c0 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.090+0000 7effa659c700 1 --2- 172.21.15.66:0/1919223729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa81099b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff980098a0 tx=0x7eff98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81abe584722534b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.090+0000 7effa559a700 1 -- 172.21.15.66:0/1919223729 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff98004160 con 0x7effa81075c0 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.090+0000 7effa559a700 1 -- 172.21.15.66:0/1919223729 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7eff980042c0 con 0x7effa81075c0 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.090+0000 7effa559a700 1 -- 172.21.15.66:0/1919223729 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff98011940 con 0x7effa81075c0 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 -- 172.21.15.66:0/1919223729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 msgr2=0x7effa81099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 --2- 172.21.15.66:0/1919223729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa81099b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7eff980098a0 tx=0x7eff98009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 -- 172.21.15.66:0/1919223729 shutdown_connections 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 --2- 172.21.15.66:0/1919223729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa81099b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 -- 172.21.15.66:0/1919223729 >> 172.21.15.66:0/1919223729 conn(0x7effa8101010 msgr2=0x7effa8103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:46.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 -- 172.21.15.66:0/1919223729 shutdown_connections 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.091+0000 7effac839700 1 -- 172.21.15.66:0/1919223729 wait complete. 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.092+0000 7effac839700 1 Processor -- start 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.092+0000 7effac839700 1 -- start start 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effac839700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa8106df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effac839700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8107330 con 0x7effa81075c0 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effa659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa8106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effa659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa8106df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40324/0 (socket says 172.21.15.66:40324) 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effa659c700 1 -- 172.21.15.66:0/1297482764 learned_addr learned my addr 172.21.15.66:0/1297482764 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effa659c700 1 -- 172.21.15.66:0/1297482764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009550 con 0x7effa81075c0 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.093+0000 7effa659c700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa8106df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff98006af0 tx=0x7eff98011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.094+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff98003bf0 con 0x7effa81075c0 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.094+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7eff9801b8b0 con 0x7effa81075c0 2023-12-11T19:23:46.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.094+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff98023830 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.094+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa8105570 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.094+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa8105a10 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.095+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7eff9801b440 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.095+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa810ac30 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.095+0000 7eff977fe700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7eff90038820 0x7eff9003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.095+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7eff9804cdf0 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.096+0000 7effa5d9b700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7eff90038820 0x7eff9003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.096+0000 7effa5d9b700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7eff90038820 0x7eff9003ace0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff9c006fd0 tx=0x7eff9c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.101+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7eff9802b530 con 0x7effa81075c0 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.311+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7effa8106200 con 0x7eff90038820 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.523+0000 7eff977fe700 1 -- 172.21.15.66:0/1297482764 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7effa8106200 con 0x7eff90038820 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7eff90038820 msgr2=0x7eff9003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7eff90038820 0x7eff9003ace0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff9c006fd0 tx=0x7eff9c006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:46.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 msgr2=0x7effa8106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa8106df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff98006af0 tx=0x7eff98011a50 comp rx=0 tx=0).stop 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 shutdown_connections 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7eff90038820 0x7eff9003ace0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 --2- 172.21.15.66:0/1297482764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa81075c0 0x7effa8106df0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 >> 172.21.15.66:0/1297482764 conn(0x7effa8101010 msgr2=0x7effa8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 shutdown_connections 2023-12-11T19:23:46.930 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:46.526+0000 7effac839700 1 -- 172.21.15.66:0/1297482764 wait complete. 2023-12-11T19:23:47.184 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-mon[97920]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:47.184 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:47.184 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:46 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:48.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:47 smithi066 ceph-mon[97920]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:48.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:47 smithi066 ceph-mon[97920]: Generating ssh key... 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.458+0000 7f2f9d596700 1 Processor -- start 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.458+0000 7f2f9d596700 1 -- start start 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.458+0000 7f2f9d596700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9807b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.458+0000 7f2f9d596700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9807b8d0 con 0x7f2f9807cea0 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.458+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9807b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.458+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9807b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40326/0 (socket says 172.21.15.66:40326) 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.459+0000 7f2f96ffd700 1 -- 172.21.15.66:0/1570104890 learned_addr learned my addr 172.21.15.66:0/1570104890 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:48.217 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.459+0000 7f2f96ffd700 1 -- 172.21.15.66:0/1570104890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9807ba10 con 0x7f2f9807cea0 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.459+0000 7f2f96ffd700 1 --2- 172.21.15.66:0/1570104890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9807b300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f880098a0 tx=0x7f2f88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b581e44e6f62f190 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.459+0000 7f2f95ffb700 1 -- 172.21.15.66:0/1570104890 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f88004160 con 0x7f2f9807cea0 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.459+0000 7f2f95ffb700 1 -- 172.21.15.66:0/1570104890 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2f880042c0 con 0x7f2f9807cea0 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.459+0000 7f2f95ffb700 1 -- 172.21.15.66:0/1570104890 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f88011940 con 0x7f2f9807cea0 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 -- 172.21.15.66:0/1570104890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 msgr2=0x7f2f9807b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 --2- 172.21.15.66:0/1570104890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9807b300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2f880098a0 tx=0x7f2f88009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 -- 172.21.15.66:0/1570104890 shutdown_connections 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 --2- 172.21.15.66:0/1570104890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9807b300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 -- 172.21.15.66:0/1570104890 >> 172.21.15.66:0/1570104890 conn(0x7f2f981020c0 msgr2=0x7f2f98104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 -- 172.21.15.66:0/1570104890 shutdown_connections 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.460+0000 7f2f9d596700 1 -- 172.21.15.66:0/1570104890 wait complete. 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.461+0000 7f2f9d596700 1 Processor -- start 2023-12-11T19:23:48.218 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.461+0000 7f2f9d596700 1 -- start start 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f9d596700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f9d596700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9819cec0 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40342/0 (socket says 172.21.15.66:40342) 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f96ffd700 1 -- 172.21.15.66:0/1065225795 learned_addr learned my addr 172.21.15.66:0/1065225795 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f96ffd700 1 -- 172.21.15.66:0/1065225795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88009550 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.462+0000 7f2f96ffd700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9819c980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2f88003f20 tx=0x7f2f88011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.463+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f880036a0 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.463+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2f8801b440 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.463+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f88023400 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.463+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9819d0c0 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.463+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9819d560 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.464+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f2f8801b5b0 con 0x7f2f9807cea0 2023-12-11T19:23:48.219 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.464+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9810ce50 con 0x7f2f9807cea0 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.464+0000 7f2f7ffff700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2f80038870 0x7f2f8003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.464+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2f8804c860 con 0x7f2f9807cea0 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.465+0000 7f2f967fc700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2f80038870 0x7f2f8003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.465+0000 7f2f967fc700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2f80038870 0x7f2f8003ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.469+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2f8801b920 con 0x7f2f9807cea0 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.683+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f98103350 con 0x7f2f80038870 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.684+0000 7f2f7ffff700 1 -- 172.21.15.66:0/1065225795 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f2f98103350 con 0x7f2f80038870 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2f80038870 msgr2=0x7f2f8003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2f80038870 0x7f2f8003ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 msgr2=0x7f2f9819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9819c980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2f88003f20 tx=0x7f2f88011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 shutdown_connections 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2f80038870 0x7f2f8003ad30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 --2- 172.21.15.66:0/1065225795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f9807cea0 0x7f2f9819c980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:48.220 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 >> 172.21.15.66:0/1065225795 conn(0x7f2f981020c0 msgr2=0x7f2f98102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:48.221 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 shutdown_connections 2023-12-11T19:23:48.221 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:47.687+0000 7f2f9d596700 1 -- 172.21.15.66:0/1065225795 wait complete. 2023-12-11T19:23:48.221 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-11T19:23:48.221 INFO:teuthology.orchestra.run.smithi066.stdout:Adding key to root@localhost authorized_keys... 2023-12-11T19:23:48.221 INFO:teuthology.orchestra.run.smithi066.stdout:Adding host smithi066... 2023-12-11T19:23:49.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:48 smithi066 ceph-mon[97920]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:51.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:50 smithi066 ceph-mon[97920]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:51.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:50 smithi066 ceph-mon[97920]: Deploying cephadm binary to smithi066 2023-12-11T19:23:54.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Added host 'smithi066' with addr '172.21.15.66' 2023-12-11T19:23:54.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.759+0000 7fb65d437700 1 Processor -- start 2023-12-11T19:23:54.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.760+0000 7fb65d437700 1 -- start start 2023-12-11T19:23:54.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.760+0000 7fb65d437700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb658106f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:54.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.760+0000 7fb65d437700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb658107550 con 0x7fb658106b60 2023-12-11T19:23:54.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.760+0000 7fb656ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb658106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.760+0000 7fb656ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb658106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40344/0 (socket says 172.21.15.66:40344) 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.760+0000 7fb656ffd700 1 -- 172.21.15.66:0/3617838182 learned_addr learned my addr 172.21.15.66:0/3617838182 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.761+0000 7fb656ffd700 1 -- 172.21.15.66:0/3617838182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658107d60 con 0x7fb658106b60 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.761+0000 7fb656ffd700 1 --2- 172.21.15.66:0/3617838182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb658106f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6480098a0 tx=0x7fb648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5aa2e1538c96c50 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.761+0000 7fb655ffb700 1 -- 172.21.15.66:0/3617838182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb648004160 con 0x7fb658106b60 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.761+0000 7fb655ffb700 1 -- 172.21.15.66:0/3617838182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb6480042c0 con 0x7fb658106b60 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.761+0000 7fb655ffb700 1 -- 172.21.15.66:0/3617838182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb648011940 con 0x7fb658106b60 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 -- 172.21.15.66:0/3617838182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 msgr2=0x7fb658106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 --2- 172.21.15.66:0/3617838182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb658106f80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6480098a0 tx=0x7fb648009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 -- 172.21.15.66:0/3617838182 shutdown_connections 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 --2- 172.21.15.66:0/3617838182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb658106f80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:54.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 -- 172.21.15.66:0/3617838182 >> 172.21.15.66:0/3617838182 conn(0x7fb6581020c0 msgr2=0x7fb658104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 -- 172.21.15.66:0/3617838182 shutdown_connections 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.762+0000 7fb65d437700 1 -- 172.21.15.66:0/3617838182 wait complete. 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.763+0000 7fb65d437700 1 Processor -- start 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.763+0000 7fb65d437700 1 -- start start 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.763+0000 7fb65d437700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb65819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.763+0000 7fb65d437700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65819cee0 con 0x7fb658106b60 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.764+0000 7fb656ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb65819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.764+0000 7fb656ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb65819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40348/0 (socket says 172.21.15.66:40348) 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.764+0000 7fb656ffd700 1 -- 172.21.15.66:0/3968431073 learned_addr learned my addr 172.21.15.66:0/3968431073 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.764+0000 7fb656ffd700 1 -- 172.21.15.66:0/3968431073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb648009550 con 0x7fb658106b60 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.764+0000 7fb656ffd700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb65819c9a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb648003f20 tx=0x7fb648011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.765+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6480036a0 con 0x7fb658106b60 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.765+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb64801b440 con 0x7fb658106b60 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.765+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65819d0e0 con 0x7fb658106b60 2023-12-11T19:23:54.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.765+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb648023400 con 0x7fb658106b60 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.765+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65819d580 con 0x7fb658106b60 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.766+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fb648003800 con 0x7fb658106b60 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.766+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65810af90 con 0x7fb658106b60 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.766+0000 7fb63ffff700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb640038870 0x7fb64003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.766+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb64804c8a0 con 0x7fb658106b60 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.767+0000 7fb6567fc700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb640038870 0x7fb64003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.767+0000 7fb6567fc700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb640038870 0x7fb64003ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb64c006fd0 tx=0x7fb64c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.771+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb64800ba30 con 0x7fb658106b60 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:48.988+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}) v1 -- 0x7fb65819fe10 con 0x7fb640038870 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.906+0000 7fb63ffff700 1 -- 172.21.15.66:0/3968431073 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fb65819fe10 con 0x7fb640038870 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb640038870 msgr2=0x7fb64003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb640038870 0x7fb64003ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb64c006fd0 tx=0x7fb64c006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 msgr2=0x7fb65819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:54.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb65819c9a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb648003f20 tx=0x7fb648011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 shutdown_connections 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb640038870 0x7fb64003ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 --2- 172.21.15.66:0/3968431073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb658106b60 0x7fb65819c9a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 >> 172.21.15.66:0/3968431073 conn(0x7fb6581020c0 msgr2=0x7fb658102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 shutdown_connections 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:53.909+0000 7fb65d437700 1 -- 172.21.15.66:0/3968431073 wait complete. 2023-12-11T19:23:54.317 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mon service... 2023-12-11T19:23:55.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:54 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:55.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:54 smithi066 ceph-mon[97920]: Added host smithi066 2023-12-11T19:23:55.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:54 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.912+0000 7feafb710700 1 Processor -- start 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.912+0000 7feafb710700 1 -- start start 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.913+0000 7feafb710700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf4109990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.913+0000 7feafb710700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf407ae50 con 0x7feaf41075a0 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.913+0000 7feaf94ac700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf4109990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.913+0000 7feaf94ac700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf4109990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35546/0 (socket says 172.21.15.66:35546) 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.913+0000 7feaf94ac700 1 -- 172.21.15.66:0/4113581478 learned_addr learned my addr 172.21.15.66:0/4113581478 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.913+0000 7feaf94ac700 1 -- 172.21.15.66:0/4113581478 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4109ed0 con 0x7feaf41075a0 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.914+0000 7feaf94ac700 1 --2- 172.21.15.66:0/4113581478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf4109990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feaf00098a0 tx=0x7feaf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577078c0d44f0b14 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.914+0000 7feaebfff700 1 -- 172.21.15.66:0/4113581478 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf0004160 con 0x7feaf41075a0 2023-12-11T19:23:55.684 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.914+0000 7feaebfff700 1 -- 172.21.15.66:0/4113581478 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7feaf00042c0 con 0x7feaf41075a0 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.914+0000 7feaebfff700 1 -- 172.21.15.66:0/4113581478 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf0011940 con 0x7feaf41075a0 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 -- 172.21.15.66:0/4113581478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 msgr2=0x7feaf4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 --2- 172.21.15.66:0/4113581478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf4109990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feaf00098a0 tx=0x7feaf0009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 -- 172.21.15.66:0/4113581478 shutdown_connections 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 --2- 172.21.15.66:0/4113581478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf4109990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 -- 172.21.15.66:0/4113581478 >> 172.21.15.66:0/4113581478 conn(0x7feaf4101010 msgr2=0x7feaf4103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 -- 172.21.15.66:0/4113581478 shutdown_connections 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.915+0000 7feafb710700 1 -- 172.21.15.66:0/4113581478 wait complete. 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.916+0000 7feafb710700 1 Processor -- start 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.916+0000 7feafb710700 1 -- start start 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.916+0000 7feafb710700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.916+0000 7feafb710700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419cea0 con 0x7feaf41075a0 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.917+0000 7feaf94ac700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:55.685 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.917+0000 7feaf94ac700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35554/0 (socket says 172.21.15.66:35554) 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.917+0000 7feaf94ac700 1 -- 172.21.15.66:0/4278747625 learned_addr learned my addr 172.21.15.66:0/4278747625 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.917+0000 7feaf94ac700 1 -- 172.21.15.66:0/4278747625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0009550 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.917+0000 7feaf94ac700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf419c960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feaf0003f20 tx=0x7feaf0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.917+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf00037d0 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.918+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7feaf001b440 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.918+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf419d0a0 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.918+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf0023590 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.918+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf419d540 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.919+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7feaf001b5b0 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.919+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf410ac10 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.919+0000 7feaea7fc700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7feae0038870 0x7feae003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.919+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feaf004c8a0 con 0x7feaf41075a0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.920+0000 7feaf8cab700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7feae0038870 0x7feae003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:55.686 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.920+0000 7feaf8cab700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7feae0038870 0x7feae003ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feae4006fd0 tx=0x7feae4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:54.925+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7feaf001b920 con 0x7feaf41075a0 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.187+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7feaf4102210 con 0x7feae0038870 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.195+0000 7feaea7fc700 1 -- 172.21.15.66:0/4278747625 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7feaf4102210 con 0x7feae0038870 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7feae0038870 msgr2=0x7feae003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7feae0038870 0x7feae003ad30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7feae4006fd0 tx=0x7feae4006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 msgr2=0x7feaf419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf419c960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feaf0003f20 tx=0x7feaf0011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 shutdown_connections 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7feae0038870 0x7feae003ad30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 --2- 172.21.15.66:0/4278747625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feaf41075a0 0x7feaf419c960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 >> 172.21.15.66:0/4278747625 conn(0x7feaf4101010 msgr2=0x7feaf4101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 shutdown_connections 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:55.197+0000 7feafb710700 1 -- 172.21.15.66:0/4278747625 wait complete. 2023-12-11T19:23:55.687 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mgr service... 2023-12-11T19:23:56.530 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:56 smithi066 ceph-mon[97920]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:56.530 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:56 smithi066 ceph-mon[97920]: Saving service mon spec with placement count:5 2023-12-11T19:23:56.530 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:56 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.255+0000 7fcd870a1700 1 Processor -- start 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.255+0000 7fcd870a1700 1 -- start start 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.256+0000 7fcd870a1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd80106f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.256+0000 7fcd870a1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80107550 con 0x7fcd80106b60 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.256+0000 7fcd84e3d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd80106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.256+0000 7fcd84e3d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd80106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35570/0 (socket says 172.21.15.66:35570) 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.256+0000 7fcd84e3d700 1 -- 172.21.15.66:0/246315218 learned_addr learned my addr 172.21.15.66:0/246315218 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.256+0000 7fcd84e3d700 1 -- 172.21.15.66:0/246315218 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd80107d60 con 0x7fcd80106b60 2023-12-11T19:23:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd84e3d700 1 --2- 172.21.15.66:0/246315218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd80106f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd740098a0 tx=0x7fcd74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc3408e907b4b79 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd7f7fe700 1 -- 172.21.15.66:0/246315218 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd74004160 con 0x7fcd80106b60 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd7f7fe700 1 -- 172.21.15.66:0/246315218 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fcd740042c0 con 0x7fcd80106b60 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd7f7fe700 1 -- 172.21.15.66:0/246315218 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd74011940 con 0x7fcd80106b60 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd870a1700 1 -- 172.21.15.66:0/246315218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 msgr2=0x7fcd80106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd870a1700 1 --2- 172.21.15.66:0/246315218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd80106f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd740098a0 tx=0x7fcd74009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd870a1700 1 -- 172.21.15.66:0/246315218 shutdown_connections 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd870a1700 1 --2- 172.21.15.66:0/246315218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd80106f80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.257+0000 7fcd870a1700 1 -- 172.21.15.66:0/246315218 >> 172.21.15.66:0/246315218 conn(0x7fcd801020c0 msgr2=0x7fcd80104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.258+0000 7fcd870a1700 1 -- 172.21.15.66:0/246315218 shutdown_connections 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.258+0000 7fcd870a1700 1 -- 172.21.15.66:0/246315218 wait complete. 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd870a1700 1 Processor -- start 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd870a1700 1 -- start start 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd870a1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd8019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:56.972 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd870a1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8019cee0 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd84e3d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd8019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd84e3d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd8019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35582/0 (socket says 172.21.15.66:35582) 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.259+0000 7fcd84e3d700 1 -- 172.21.15.66:0/1348908581 learned_addr learned my addr 172.21.15.66:0/1348908581 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd84e3d700 1 -- 172.21.15.66:0/1348908581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74009550 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd84e3d700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd8019c9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd74003f20 tx=0x7fcd74011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd740036a0 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fcd7401b440 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8019d0e0 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd74023400 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.260+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8019d580 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.261+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fcd74003800 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.262+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8010af90 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.262+0000 7fcd7dffb700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fcd70038870 0x7fcd7003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.262+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcd7404c8a0 con 0x7fcd80106b60 2023-12-11T19:23:56.973 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.262+0000 7fcd7ffff700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fcd70038870 0x7fcd7003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.263+0000 7fcd7ffff700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fcd70038870 0x7fcd7003ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c006fd0 tx=0x7fcd6c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.267+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcd7400ba30 con 0x7fcd80106b60 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.495+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcd8019fe10 con 0x7fcd70038870 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.503+0000 7fcd7dffb700 1 -- 172.21.15.66:0/1348908581 <== mgr.14118 v2:172.21.15.66:6800/4268281106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcd8019fe10 con 0x7fcd70038870 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fcd70038870 msgr2=0x7fcd7003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fcd70038870 0x7fcd7003ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c006fd0 tx=0x7fcd6c006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 msgr2=0x7fcd8019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd8019c9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd74003f20 tx=0x7fcd74011f00 comp rx=0 tx=0).stop 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 shutdown_connections 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fcd70038870 0x7fcd7003ad30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 --2- 172.21.15.66:0/1348908581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd80106b60 0x7fcd8019c9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.505+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 >> 172.21.15.66:0/1348908581 conn(0x7fcd801020c0 msgr2=0x7fcd80102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.506+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 shutdown_connections 2023-12-11T19:23:56.974 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:56.506+0000 7fcd870a1700 1 -- 172.21.15.66:0/1348908581 wait complete. 2023-12-11T19:23:57.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:57 smithi066 ceph-mon[97920]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:23:57.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:57 smithi066 ceph-mon[97920]: Saving service mgr spec with placement count:2 2023-12-11T19:23:57.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:57 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:57.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:57 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:57.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:57 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:23:58.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.539+0000 7f0d97248700 1 Processor -- start 2023-12-11T19:23:58.168 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.539+0000 7f0d97248700 1 -- start start 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.540+0000 7f0d97248700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.540+0000 7f0d97248700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9007b890 con 0x7f0d9007ce60 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.540+0000 7f0d94fe4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.540+0000 7f0d94fe4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9007b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35588/0 (socket says 172.21.15.66:35588) 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.540+0000 7f0d94fe4700 1 -- 172.21.15.66:0/1095965419 learned_addr learned my addr 172.21.15.66:0/1095965419 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.540+0000 7f0d94fe4700 1 -- 172.21.15.66:0/1095965419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9007b9d0 con 0x7f0d9007ce60 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.541+0000 7f0d94fe4700 1 --2- 172.21.15.66:0/1095965419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9007b2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d840098a0 tx=0x7f0d84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad27fc0d83b0abb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.541+0000 7f0d8f7fe700 1 -- 172.21.15.66:0/1095965419 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d84004160 con 0x7f0d9007ce60 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.541+0000 7f0d8f7fe700 1 -- 172.21.15.66:0/1095965419 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d840042c0 con 0x7f0d9007ce60 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.541+0000 7f0d8f7fe700 1 -- 172.21.15.66:0/1095965419 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d840118e0 con 0x7f0d9007ce60 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 -- 172.21.15.66:0/1095965419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 msgr2=0x7f0d9007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:58.169 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 --2- 172.21.15.66:0/1095965419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9007b2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d840098a0 tx=0x7f0d84009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 -- 172.21.15.66:0/1095965419 shutdown_connections 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 --2- 172.21.15.66:0/1095965419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9007b2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 -- 172.21.15.66:0/1095965419 >> 172.21.15.66:0/1095965419 conn(0x7f0d901020c0 msgr2=0x7f0d90104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 -- 172.21.15.66:0/1095965419 shutdown_connections 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.542+0000 7f0d97248700 1 -- 172.21.15.66:0/1095965419 wait complete. 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.543+0000 7f0d97248700 1 Processor -- start 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.543+0000 7f0d97248700 1 -- start start 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.543+0000 7f0d97248700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.543+0000 7f0d97248700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9019cf80 con 0x7f0d9007ce60 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.544+0000 7f0d94fe4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.544+0000 7f0d94fe4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35596/0 (socket says 172.21.15.66:35596) 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.544+0000 7f0d94fe4700 1 -- 172.21.15.66:0/486299932 learned_addr learned my addr 172.21.15.66:0/486299932 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:58.170 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.544+0000 7f0d94fe4700 1 -- 172.21.15.66:0/486299932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d84009550 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.544+0000 7f0d94fe4700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9019ca40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d84003f70 tx=0x7f0d840119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.545+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d840036a0 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.545+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d8401b440 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.545+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d84023400 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.545+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9019d180 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.545+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9019d620 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.546+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f0d8401b5b0 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.546+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9010cf40 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.547+0000 7f0d8dffb700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f0d800388c0 0x7f0d8003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.547+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d8404c810 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.547+0000 7f0d8ffff700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f0d800388c0 0x7f0d8003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.548+0000 7f0d8ffff700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f0d800388c0 0x7f0d8003ad80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006fd0 tx=0x7f0d7c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.551+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0d84019840 con 0x7f0d9007ce60 2023-12-11T19:23:58.171 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.768+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0d9019fe60 con 0x7f0d9007ce60 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.771+0000 7f0d8dffb700 1 -- 172.21.15.66:0/486299932 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0d84019840 con 0x7f0d9007ce60 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f0d800388c0 msgr2=0x7f0d8003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f0d800388c0 0x7f0d8003ad80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006fd0 tx=0x7f0d7c006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 msgr2=0x7f0d9019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9019ca40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0d84003f70 tx=0x7f0d840119a0 comp rx=0 tx=0).stop 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 shutdown_connections 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f0d800388c0 0x7f0d8003ad80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 --2- 172.21.15.66:0/486299932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d9007ce60 0x7f0d9019ca40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 >> 172.21.15.66:0/486299932 conn(0x7f0d901020c0 msgr2=0x7f0d90102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.774+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 shutdown_connections 2023-12-11T19:23:58.172 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:57.775+0000 7f0d97248700 1 -- 172.21.15.66:0/486299932 wait complete. 2023-12-11T19:23:59.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:58 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/486299932' entity='client.admin' 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.710+0000 7f2dd467e700 1 Processor -- start 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.710+0000 7f2dd467e700 1 -- start start 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.710+0000 7f2dd467e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc109a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.710+0000 7f2dd467e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc07add0 con 0x7f2dcc107620 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.711+0000 7f2dd241a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.711+0000 7f2dd241a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc109a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35598/0 (socket says 172.21.15.66:35598) 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.711+0000 7f2dd241a700 1 -- 172.21.15.66:0/2657549446 learned_addr learned my addr 172.21.15.66:0/2657549446 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.711+0000 7f2dd241a700 1 -- 172.21.15.66:0/2657549446 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc07af10 con 0x7f2dcc107620 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.711+0000 7f2dd241a700 1 --2- 172.21.15.66:0/2657549446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc109a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098a0 tx=0x7f2dc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ede2fbf5ea98fa4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd1418700 1 -- 172.21.15.66:0/2657549446 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8004160 con 0x7f2dcc107620 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd1418700 1 -- 172.21.15.66:0/2657549446 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dc80042c0 con 0x7f2dcc107620 2023-12-11T19:23:59.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd1418700 1 -- 172.21.15.66:0/2657549446 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dc80118e0 con 0x7f2dcc107620 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 -- 172.21.15.66:0/2657549446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 msgr2=0x7f2dcc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 --2- 172.21.15.66:0/2657549446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc109a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098a0 tx=0x7f2dc8009bb0 comp rx=0 tx=0).stop 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 -- 172.21.15.66:0/2657549446 shutdown_connections 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 --2- 172.21.15.66:0/2657549446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc109a10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 -- 172.21.15.66:0/2657549446 >> 172.21.15.66:0/2657549446 conn(0x7f2dcc101010 msgr2=0x7f2dcc103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 -- 172.21.15.66:0/2657549446 shutdown_connections 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.712+0000 7f2dd467e700 1 -- 172.21.15.66:0/2657549446 wait complete. 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.713+0000 7f2dd467e700 1 Processor -- start 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.714+0000 7f2dd467e700 1 -- start start 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.714+0000 7f2dd467e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.714+0000 7f2dd467e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19ad30 con 0x7f2dcc107620 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.714+0000 7f2dd241a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:59.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.714+0000 7f2dd241a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35604/0 (socket says 172.21.15.66:35604) 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.714+0000 7f2dd241a700 1 -- 172.21.15.66:0/3522200091 learned_addr learned my addr 172.21.15.66:0/3522200091 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dd241a700 1 -- 172.21.15.66:0/3522200091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc8009550 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dd241a700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc19a7f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8003f70 tx=0x7f2dc80119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dc80036a0 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2dc801b440 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8023400 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc19af30 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.715+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc19b3d0 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.716+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f2dc801b5b0 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.716+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc10ac60 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.717+0000 7f2dc37fe700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2db8038870 0x7f2db803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.717+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2dc804c750 con 0x7f2dcc107620 2023-12-11T19:23:59.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.717+0000 7f2dd1c19700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2db8038870 0x7f2db803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.718+0000 7f2dd1c19700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2db8038870 0x7f2db803ad30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc006fd0 tx=0x7f2dbc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.722+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2dc8019840 con 0x7f2dcc107620 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.941+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f2dcc10aea0 con 0x7f2dcc107620 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.945+0000 7f2dc37fe700 1 -- 172.21.15.66:0/3522200091 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f2dc8019840 con 0x7f2dcc107620 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2db8038870 msgr2=0x7f2db803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2db8038870 0x7f2db803ad30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc006fd0 tx=0x7f2dbc006e40 comp rx=0 tx=0).stop 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 msgr2=0x7f2dcc19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc19a7f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8003f70 tx=0x7f2dc80119a0 comp rx=0 tx=0).stop 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 shutdown_connections 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f2db8038870 0x7f2db803ad30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 --2- 172.21.15.66:0/3522200091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc107620 0x7f2dcc19a7f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.947+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 >> 172.21.15.66:0/3522200091 conn(0x7f2dcc101010 msgr2=0x7f2dcc101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.948+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 shutdown_connections 2023-12-11T19:23:59.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:23:58.948+0000 7f2dd467e700 1 -- 172.21.15.66:0/3522200091 wait complete. 2023-12-11T19:23:59.560 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling the dashboard module... 2023-12-11T19:24:00.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:59 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3522200091' entity='client.admin' 2023-12-11T19:24:00.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:23:59 smithi066 ceph-mon[97920]: from='mgr.14118 172.21.15.66:0/2126627170' entity='mgr.x' 2023-12-11T19:24:01.299 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:00 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3163043327' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-11T19:24:01.300 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: ignoring --setuser ceph since I am not root 2023-12-11T19:24:01.300 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: ignoring --setgroup ceph since I am not root 2023-12-11T19:24:01.300 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:01.297+0000 7f9fdaa35200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T19:24:01.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.119+0000 7f12cc52f700 1 Processor -- start 2023-12-11T19:24:01.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.120+0000 7f12cc52f700 1 -- start start 2023-12-11T19:24:01.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.120+0000 7f12cc52f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c407b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.120+0000 7f12cc52f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c407b870 con 0x7f12c407ce40 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.120+0000 7f12ca2cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c407b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.120+0000 7f12ca2cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c407b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35616/0 (socket says 172.21.15.66:35616) 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.120+0000 7f12ca2cb700 1 -- 172.21.15.66:0/3878142462 learned_addr learned my addr 172.21.15.66:0/3878142462 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.121+0000 7f12ca2cb700 1 -- 172.21.15.66:0/3878142462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c407b9b0 con 0x7f12c407ce40 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.121+0000 7f12ca2cb700 1 --2- 172.21.15.66:0/3878142462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c407b2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12c00098a0 tx=0x7f12c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80864ce9e039c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.121+0000 7f12c92c9700 1 -- 172.21.15.66:0/3878142462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c0004160 con 0x7f12c407ce40 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.121+0000 7f12c92c9700 1 -- 172.21.15.66:0/3878142462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12c00042c0 con 0x7f12c407ce40 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.121+0000 7f12c92c9700 1 -- 172.21.15.66:0/3878142462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c00118e0 con 0x7f12c407ce40 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 -- 172.21.15.66:0/3878142462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 msgr2=0x7f12c407b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 --2- 172.21.15.66:0/3878142462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c407b2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12c00098a0 tx=0x7f12c0009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 -- 172.21.15.66:0/3878142462 shutdown_connections 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 --2- 172.21.15.66:0/3878142462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c407b2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:01.447 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 -- 172.21.15.66:0/3878142462 >> 172.21.15.66:0/3878142462 conn(0x7f12c41020c0 msgr2=0x7f12c4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 -- 172.21.15.66:0/3878142462 shutdown_connections 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.122+0000 7f12cc52f700 1 -- 172.21.15.66:0/3878142462 wait complete. 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.124+0000 7f12cc52f700 1 Processor -- start 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.124+0000 7f12cc52f700 1 -- start start 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.124+0000 7f12cc52f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.124+0000 7f12cc52f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c419cf40 con 0x7f12c407ce40 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12ca2cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12ca2cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35618/0 (socket says 172.21.15.66:35618) 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12ca2cb700 1 -- 172.21.15.66:0/3163043327 learned_addr learned my addr 172.21.15.66:0/3163043327 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12ca2cb700 1 -- 172.21.15.66:0/3163043327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0009550 con 0x7f12c407ce40 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12ca2cb700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c419ca00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12c0003f70 tx=0x7f12c00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c00036a0 con 0x7f12c407ce40 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12c001b440 con 0x7f12c407ce40 2023-12-11T19:24:01.448 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.125+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c419d140 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.126+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c0023400 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.126+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c419d5e0 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.127+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c410cf00 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.127+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f12c001b5b0 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.128+0000 7f12bb7fe700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f12b0038820 0x7f12b003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.128+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f12c004c890 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.128+0000 7f12c9aca700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f12b0038820 0x7f12b003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.128+0000 7f12c9aca700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f12b0038820 0x7f12b003ace0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006fd0 tx=0x7f12b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.133+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f12c0019840 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.435+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f12c410d140 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.948+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f12c0019840 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.950+0000 7f12bb7fe700 1 -- 172.21.15.66:0/3163043327 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f12c0003800 con 0x7f12c407ce40 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f12b0038820 msgr2=0x7f12b003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f12b0038820 0x7f12b003ace0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006fd0 tx=0x7f12b4006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:01.449 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 msgr2=0x7f12c419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c419ca00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12c0003f70 tx=0x7f12c00119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 shutdown_connections 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f12b0038820 0x7f12b003ace0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 --2- 172.21.15.66:0/3163043327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12c407ce40 0x7f12c419ca00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 >> 172.21.15.66:0/3163043327 conn(0x7f12c41020c0 msgr2=0x7f12c4102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.953+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 shutdown_connections 2023-12-11T19:24:01.450 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:00.954+0000 7f12cc52f700 1 -- 172.21.15.66:0/3163043327 wait complete. 2023-12-11T19:24:01.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:01.415+0000 7f9fdaa35200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T19:24:01.933 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:01.561+0000 7f9fdaa35200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T19:24:01.933 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:01.689+0000 7f9fdaa35200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T19:24:01.933 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:01.930+0000 7f9fdaa35200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T19:24:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3163043327' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-11T19:24:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:01 smithi066 ceph-mon[97920]: mgrmap e9: x(active, since 19s) 2023-12-11T19:24:02.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:02.426+0000 7f9fdaa35200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T19:24:02.741 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:24:02.747 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.001+0000 7fb09a6d2700 1 Processor -- start 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.002+0000 7fb09a6d2700 1 -- start start 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.002+0000 7fb09a6d2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.002+0000 7fb09a6d2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09407b8b0 con 0x7fb09407ce80 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.002+0000 7fb093fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb093fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09407b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35630/0 (socket says 172.21.15.66:35630) 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb093fff700 1 -- 172.21.15.66:0/1206575516 learned_addr learned my addr 172.21.15.66:0/1206575516 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb093fff700 1 -- 172.21.15.66:0/1206575516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09407b9f0 con 0x7fb09407ce80 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb093fff700 1 --2- 172.21.15.66:0/1206575516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09407b2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0840098a0 tx=0x7fb084009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3a9459f8dcb370 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:02.748 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb092ffd700 1 -- 172.21.15.66:0/1206575516 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb084004160 con 0x7fb09407ce80 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb092ffd700 1 -- 172.21.15.66:0/1206575516 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0840042c0 con 0x7fb09407ce80 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.003+0000 7fb092ffd700 1 -- 172.21.15.66:0/1206575516 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0840118e0 con 0x7fb09407ce80 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.004+0000 7fb09a6d2700 1 -- 172.21.15.66:0/1206575516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 msgr2=0x7fb09407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.004+0000 7fb09a6d2700 1 --2- 172.21.15.66:0/1206575516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09407b2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0840098a0 tx=0x7fb084009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.004+0000 7fb09a6d2700 1 -- 172.21.15.66:0/1206575516 shutdown_connections 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.004+0000 7fb09a6d2700 1 --2- 172.21.15.66:0/1206575516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09407b2e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.004+0000 7fb09a6d2700 1 -- 172.21.15.66:0/1206575516 >> 172.21.15.66:0/1206575516 conn(0x7fb0941020c0 msgr2=0x7fb094104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.004+0000 7fb09a6d2700 1 -- 172.21.15.66:0/1206575516 shutdown_connections 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.005+0000 7fb09a6d2700 1 -- 172.21.15.66:0/1206575516 wait complete. 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb09a6d2700 1 Processor -- start 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb09a6d2700 1 -- start start 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb09a6d2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb09a6d2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09419cfb0 con 0x7fb09407ce80 2023-12-11T19:24:02.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb093fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb093fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46660/0 (socket says 172.21.15.66:46660) 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.006+0000 7fb093fff700 1 -- 172.21.15.66:0/812492543 learned_addr learned my addr 172.21.15.66:0/812492543 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.007+0000 7fb093fff700 1 -- 172.21.15.66:0/812492543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084009550 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.007+0000 7fb093fff700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09419ca70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb084006af0 tx=0x7fb084011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.007+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0840037d0 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.007+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb08401b440 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.007+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb084023400 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.007+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09419d1b0 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.008+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09419d650 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.009+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fb084023670 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.009+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09410cf20 con 0x7fb09407ce80 2023-12-11T19:24:02.750 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.009+0000 7fb0917fa700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 0x7fb07c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.009+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb08404ca50 con 0x7fb09407ce80 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.009+0000 7fb0937fe700 1 -- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 msgr2=0x7fb07c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.009+0000 7fb0937fe700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 0x7fb07c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.014+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb084018350 con 0x7fb09407ce80 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.210+0000 7fb0937fe700 1 -- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 msgr2=0x7fb07c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.210+0000 7fb0937fe700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 0x7fb07c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.285+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb09419fee0 con 0x7fb09407ce80 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.286+0000 7fb0917fa700 1 -- 172.21.15.66:0/812492543 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb08402bb40 con 0x7fb09407ce80 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 msgr2=0x7fb07c03ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 0x7fb07c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 msgr2=0x7fb09419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09419ca70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb084006af0 tx=0x7fb084011ed0 comp rx=0 tx=0).stop 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 shutdown_connections 2023-12-11T19:24:02.751 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7fb07c038870 0x7fb07c03ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:02.752 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 --2- 172.21.15.66:0/812492543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb09407ce80 0x7fb09419ca70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:02.752 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 >> 172.21.15.66:0/812492543 conn(0x7fb0941020c0 msgr2=0x7fb094102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:02.752 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 shutdown_connections 2023-12-11T19:24:02.752 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:02.288+0000 7fb09a6d2700 1 -- 172.21.15.66:0/812492543 wait complete. 2023-12-11T19:24:02.752 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2023-12-11T19:24:02.752 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 9... 2023-12-11T19:24:02.874 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:02.601+0000 7f9fdaa35200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T19:24:02.875 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:02.725+0000 7f9fdaa35200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T19:24:03.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:02 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/812492543' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-11T19:24:03.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:02.872+0000 7f9fdaa35200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T19:24:03.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:03 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:03.482+0000 7f9fdaa35200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T19:24:04.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:04 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:04.390+0000 7f9fdaa35200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T19:24:04.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:04 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:04.502+0000 7f9fdaa35200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T19:24:05.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:05.519+0000 7f9fdaa35200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:24:05.813 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:05.645+0000 7f9fdaa35200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T19:24:06.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:05.883+0000 7f9fdaa35200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:24:06.769 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:06.438+0000 7f9fdaa35200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T19:24:06.769 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:06.561+0000 7f9fdaa35200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T19:24:07.063 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:06.767+0000 7f9fdaa35200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T19:24:07.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:07 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:07.132+0000 7f9fdaa35200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T19:24:08.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:08.120+0000 7f9fdaa35200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T19:24:08.563 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:08.305+0000 7f9fdaa35200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T19:24:09.024 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:08.572+0000 7f9fdaa35200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T19:24:09.024 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:08.726+0000 7f9fdaa35200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T19:24:09.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:09 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:09.022+0000 7f9fdaa35200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T19:24:10.565 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:24:10.187+0000 7f9fdaa35200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: Active manager daemon x restarted 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: Activating manager daemon x 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: osdmap e3: 0 total, 0 up, 0 in 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: mgrmap e10: x(active, starting, since 0.00648093s) 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T19:24:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: Manager daemon x is now available 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T19:24:11.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T19:24:11.553 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.320+0000 7f6e56f0e700 1 Processor -- start 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.320+0000 7f6e56f0e700 1 -- start start 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.320+0000 7f6e56f0e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e50106f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.320+0000 7f6e56f0e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e50107550 con 0x7f6e50106b60 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.321+0000 7f6e54caa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e50106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.321+0000 7f6e54caa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e50106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46664/0 (socket says 172.21.15.66:46664) 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.321+0000 7f6e54caa700 1 -- 172.21.15.66:0/352748331 learned_addr learned my addr 172.21.15.66:0/352748331 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:11.554 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.321+0000 7f6e54caa700 1 -- 172.21.15.66:0/352748331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50107d60 con 0x7f6e50106b60 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.322+0000 7f6e54caa700 1 --2- 172.21.15.66:0/352748331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e50106f80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098a0 tx=0x7f6e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da44968a30e5369 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.322+0000 7f6e4f7fe700 1 -- 172.21.15.66:0/352748331 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e44004160 con 0x7f6e50106b60 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.322+0000 7f6e4f7fe700 1 -- 172.21.15.66:0/352748331 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e440042c0 con 0x7f6e50106b60 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.322+0000 7f6e4f7fe700 1 -- 172.21.15.66:0/352748331 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e440118e0 con 0x7f6e50106b60 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.322+0000 7f6e56f0e700 1 -- 172.21.15.66:0/352748331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 msgr2=0x7f6e50106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.322+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/352748331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e50106f80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098a0 tx=0x7f6e44009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.323+0000 7f6e56f0e700 1 -- 172.21.15.66:0/352748331 shutdown_connections 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.323+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/352748331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e50106f80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.323+0000 7f6e56f0e700 1 -- 172.21.15.66:0/352748331 >> 172.21.15.66:0/352748331 conn(0x7f6e501020c0 msgr2=0x7f6e50104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.323+0000 7f6e56f0e700 1 -- 172.21.15.66:0/352748331 shutdown_connections 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.323+0000 7f6e56f0e700 1 -- 172.21.15.66:0/352748331 wait complete. 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.324+0000 7f6e56f0e700 1 Processor -- start 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.324+0000 7f6e56f0e700 1 -- start start 2023-12-11T19:24:11.555 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.325+0000 7f6e56f0e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e5019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.325+0000 7f6e56f0e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019ada0 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.325+0000 7f6e54caa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e5019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.325+0000 7f6e54caa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e5019a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46672/0 (socket says 172.21.15.66:46672) 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.325+0000 7f6e54caa700 1 -- 172.21.15.66:0/1022515839 learned_addr learned my addr 172.21.15.66:0/1022515839 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e54caa700 1 -- 172.21.15.66:0/1022515839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44009550 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e54caa700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e5019a860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e44003f70 tx=0x7f6e440119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e440036a0 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e4401b440 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5019afa0 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e44023400 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.326+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5019b440 con 0x7f6e50106b60 2023-12-11T19:24:11.556 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.328+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f6e4401b5b0 con 0x7f6e50106b60 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.328+0000 7f6e4dffb700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.328+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 --> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e4003b440 con 0x7f6e40038870 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.328+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6e4404c7f0 con 0x7f6e50106b60 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.328+0000 7f6e4ffff700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.328+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.529+0000 7f6e4ffff700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.529+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.930+0000 7f6e4ffff700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:03.930+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:04.731+0000 7f6e4ffff700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:04.731+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-11T19:24:11.557 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:06.333+0000 7f6e4ffff700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:06.333+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:09.536+0000 7f6e4ffff700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/4268281106 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:09.536+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:10.194+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mgrmap(e 10) v1 ==== 45360+0+0 (secure 0 0 0) 0x7f6e44023680 con 0x7f6e50106b60 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:10.194+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 msgr2=0x7f6e4003ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:10.194+0000 7f6e4dffb700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.196+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7f6e44003800 con 0x7f6e50106b60 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.197+0000 7f6e4dffb700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6e4003c1b0 0x7f6e4003e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.197+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e4003b440 con 0x7f6e4003c1b0 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.197+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6e4003c1b0 0x7f6e4003e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.198+0000 7f6e4ffff700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6e4003c1b0 0x7f6e4003e5a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c003a10 tx=0x7f6e3c0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:11.558 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.199+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f6e4003b440 con 0x7f6e4003c1b0 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6e50062c00 con 0x7f6e4003c1b0 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e4dffb700 1 -- 172.21.15.66:0/1022515839 <== mgr.14150 v2:172.21.15.66:6800/775613503 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f6e50062c00 con 0x7f6e4003c1b0 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6e4003c1b0 msgr2=0x7f6e4003e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6e4003c1b0 0x7f6e4003e5a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c003a10 tx=0x7f6e3c0092b0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 msgr2=0x7f6e5019a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e5019a860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e44003f70 tx=0x7f6e440119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 shutdown_connections 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6e4003c1b0 0x7f6e4003e5a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e50106b60 0x7f6e5019a860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 --2- 172.21.15.66:0/1022515839 >> [v2:172.21.15.66:6800/4268281106,v1:172.21.15.66:6801/4268281106] conn(0x7f6e40038870 0x7f6e4003ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.207+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 >> 172.21.15.66:0/1022515839 conn(0x7f6e501020c0 msgr2=0x7f6e50102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.208+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 shutdown_connections 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:11.208+0000 7f6e56f0e700 1 -- 172.21.15.66:0/1022515839 wait complete. 2023-12-11T19:24:11.559 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 9 is available 2023-12-11T19:24:11.560 INFO:teuthology.orchestra.run.smithi066.stdout:Generating a dashboard self-signed certificate... 2023-12-11T19:24:11.895 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:11 smithi066 ceph-mon[97920]: [11/Dec/2023:19:24:10] ENGINE Bus STARTING 2023-12-11T19:24:11.895 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:11 smithi066 ceph-mon[97920]: [11/Dec/2023:19:24:11] ENGINE Serving on http://172.21.15.66:8765 2023-12-11T19:24:11.895 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:11 smithi066 ceph-mon[97920]: mgrmap e11: x(active, since 1.01004s) 2023-12-11T19:24:12.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: [11/Dec/2023:19:24:11] ENGINE Serving on https://172.21.15.66:7150 2023-12-11T19:24:12.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: [11/Dec/2023:19:24:11] ENGINE Bus STARTED 2023-12-11T19:24:12.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-11T19:24:12.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-11T19:24:12.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:12.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:12.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4c58ad700 1 Processor -- start 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4c58ad700 1 -- start start 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4c58ad700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c01099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4c58ad700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c007ae10 con 0x7ff4c01075e0 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c01099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c01099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39912/0 (socket says 172.21.15.66:39912) 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4beffd700 1 -- 172.21.15.66:0/1757735438 learned_addr learned my addr 172.21.15.66:0/1757735438 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.160+0000 7ff4beffd700 1 -- 172.21.15.66:0/1757735438 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0109f10 con 0x7ff4c01075e0 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.161+0000 7ff4beffd700 1 --2- 172.21.15.66:0/1757735438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c01099d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00098a0 tx=0x7ff4b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1e9190cf550f80 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.161+0000 7ff4bdffb700 1 -- 172.21.15.66:0/1757735438 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4b0004160 con 0x7ff4c01075e0 2023-12-11T19:24:12.841 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.161+0000 7ff4bdffb700 1 -- 172.21.15.66:0/1757735438 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4b00042c0 con 0x7ff4c01075e0 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.161+0000 7ff4bdffb700 1 -- 172.21.15.66:0/1757735438 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4b00118e0 con 0x7ff4c01075e0 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 -- 172.21.15.66:0/1757735438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 msgr2=0x7ff4c01099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 --2- 172.21.15.66:0/1757735438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c01099d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00098a0 tx=0x7ff4b0009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 -- 172.21.15.66:0/1757735438 shutdown_connections 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 --2- 172.21.15.66:0/1757735438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c01099d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 -- 172.21.15.66:0/1757735438 >> 172.21.15.66:0/1757735438 conn(0x7ff4c0101010 msgr2=0x7ff4c0103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 -- 172.21.15.66:0/1757735438 shutdown_connections 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.162+0000 7ff4c58ad700 1 -- 172.21.15.66:0/1757735438 wait complete. 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.163+0000 7ff4c58ad700 1 Processor -- start 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.163+0000 7ff4c58ad700 1 -- start start 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.163+0000 7ff4c58ad700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.163+0000 7ff4c58ad700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c019cf90 con 0x7ff4c01075e0 2023-12-11T19:24:12.842 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.164+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.164+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39920/0 (socket says 172.21.15.66:39920) 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.164+0000 7ff4beffd700 1 -- 172.21.15.66:0/3499406253 learned_addr learned my addr 172.21.15.66:0/3499406253 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.164+0000 7ff4beffd700 1 -- 172.21.15.66:0/3499406253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0009550 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.164+0000 7ff4beffd700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c019ca50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0003f70 tx=0x7ff4b00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.165+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4b00036a0 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.165+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4b001b440 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.165+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c019d190 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.165+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4b0023400 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.165+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c019d630 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.166+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c010ac40 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.166+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7ff4b0023680 con 0x7ff4c01075e0 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.167+0000 7ff4c48ab700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff4a8038740 0x7ff4a803ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:12.843 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.167+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff4b004c6e0 con 0x7ff4c01075e0 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.167+0000 7ff4be7fc700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff4a8038740 0x7ff4a803ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.168+0000 7ff4be7fc700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff4a8038740 0x7ff4a803ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4006fd0 tx=0x7ff4b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.172+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff4b002bd20 con 0x7ff4c01075e0 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.393+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4c019fec0 con 0x7ff4a8038740 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.476+0000 7ff4c48ab700 1 -- 172.21.15.66:0/3499406253 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff4c019fec0 con 0x7ff4a8038740 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff4a8038740 msgr2=0x7ff4a803ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff4a8038740 0x7ff4a803ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4006fd0 tx=0x7ff4b4006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 msgr2=0x7ff4c019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c019ca50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0003f70 tx=0x7ff4b00119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 shutdown_connections 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff4a8038740 0x7ff4a803ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:12.844 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 --2- 172.21.15.66:0/3499406253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4c01075e0 0x7ff4c019ca50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:12.845 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 >> 172.21.15.66:0/3499406253 conn(0x7ff4c0101010 msgr2=0x7ff4c0101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:12.845 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 shutdown_connections 2023-12-11T19:24:12.845 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:12.479+0000 7ff4c58ad700 1 -- 172.21.15.66:0/3499406253 wait complete. 2023-12-11T19:24:12.845 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial admin user... 2023-12-11T19:24:14.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:13 smithi066 ceph-mon[97920]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:14.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:13 smithi066 ceph-mon[97920]: mgrmap e12: x(active, since 2s) 2023-12-11T19:24:14.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KhK0Xkb/LAugKtyQ57JDMu9GbI/q/OfyILiHZBMsxLKtTwStYab8y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1702322653, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-11T19:24:14.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.395+0000 7f7c2f100700 1 Processor -- start 2023-12-11T19:24:14.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.396+0000 7f7c2f100700 1 -- start start 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.396+0000 7f7c2f100700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c281099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.396+0000 7f7c2f100700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2807ae30 con 0x7f7c281075c0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.396+0000 7f7c2ce9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c281099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.396+0000 7f7c2ce9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c281099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39936/0 (socket says 172.21.15.66:39936) 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.396+0000 7f7c2ce9c700 1 -- 172.21.15.66:0/439007924 learned_addr learned my addr 172.21.15.66:0/439007924 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.397+0000 7f7c2ce9c700 1 -- 172.21.15.66:0/439007924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28109ef0 con 0x7f7c281075c0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.397+0000 7f7c2ce9c700 1 --2- 172.21.15.66:0/439007924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c281099b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0098a0 tx=0x7f7c1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58b39af38c7493c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.397+0000 7f7c277fe700 1 -- 172.21.15.66:0/439007924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c1c004160 con 0x7f7c281075c0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.397+0000 7f7c277fe700 1 -- 172.21.15.66:0/439007924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c1c0042c0 con 0x7f7c281075c0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.397+0000 7f7c277fe700 1 -- 172.21.15.66:0/439007924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c1c0118e0 con 0x7f7c281075c0 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 -- 172.21.15.66:0/439007924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 msgr2=0x7f7c281099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 --2- 172.21.15.66:0/439007924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c281099b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0098a0 tx=0x7f7c1c009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:14.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 -- 172.21.15.66:0/439007924 shutdown_connections 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 --2- 172.21.15.66:0/439007924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c281099b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 -- 172.21.15.66:0/439007924 >> 172.21.15.66:0/439007924 conn(0x7f7c28101010 msgr2=0x7f7c28103430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 -- 172.21.15.66:0/439007924 shutdown_connections 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.398+0000 7f7c2f100700 1 -- 172.21.15.66:0/439007924 wait complete. 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.400+0000 7f7c2f100700 1 Processor -- start 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.400+0000 7f7c2f100700 1 -- start start 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.400+0000 7f7c2f100700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c28106df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.400+0000 7f7c2f100700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28107330 con 0x7f7c281075c0 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.401+0000 7f7c2ce9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c28106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.401+0000 7f7c2ce9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c28106df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39948/0 (socket says 172.21.15.66:39948) 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.401+0000 7f7c2ce9c700 1 -- 172.21.15.66:0/80151550 learned_addr learned my addr 172.21.15.66:0/80151550 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.401+0000 7f7c2ce9c700 1 -- 172.21.15.66:0/80151550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c009550 con 0x7f7c281075c0 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.401+0000 7f7c2ce9c700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c28106df0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c003f70 tx=0x7f7c1c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:14.527 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.401+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c1c0036a0 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.402+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c28105570 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.402+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c1c01b440 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.402+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c1c023400 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.402+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c28105a10 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.403+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f7c1c01b5b0 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.403+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2810ac70 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.403+0000 7f7c25ffb700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7c18038870 0x7f7c1803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.403+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c1c04c9d0 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.404+0000 7f7c27fff700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7c18038870 0x7f7c1803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.404+0000 7f7c27fff700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7c18038870 0x7f7c1803ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c14006fd0 tx=0x7f7c14006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.408+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7c1c019a20 con 0x7f7c281075c0 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.661+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7c28062a20 con 0x7f7c18038870 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.913+0000 7f7c25ffb700 1 -- 172.21.15.66:0/80151550 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7c28062a20 con 0x7f7c18038870 2023-12-11T19:24:14.528 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7c18038870 msgr2=0x7f7c1803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7c18038870 0x7f7c1803ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c14006fd0 tx=0x7f7c14006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 msgr2=0x7f7c28106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c28106df0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c003f70 tx=0x7f7c1c0119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 shutdown_connections 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7c18038870 0x7f7c1803ad30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 --2- 172.21.15.66:0/80151550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c281075c0 0x7f7c28106df0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 >> 172.21.15.66:0/80151550 conn(0x7f7c28101010 msgr2=0x7f7c28101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 shutdown_connections 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:13.916+0000 7f7c2f100700 1 -- 172.21.15.66:0/80151550 wait complete. 2023-12-11T19:24:14.529 INFO:teuthology.orchestra.run.smithi066.stdout:Fetching dashboard port number... 2023-12-11T19:24:15.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:14 smithi066 ceph-mon[97920]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:15.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:15.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:16.277 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:15 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/831044685' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-11T19:24:16.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 8443 2023-12-11T19:24:16.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.080+0000 7fef73efc700 1 Processor -- start 2023-12-11T19:24:16.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.080+0000 7fef73efc700 1 -- start start 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.081+0000 7fef73efc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.081+0000 7fef73efc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c109760 con 0x7fef6c108d70 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.081+0000 7fef71c98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.081+0000 7fef71c98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39964/0 (socket says 172.21.15.66:39964) 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.081+0000 7fef71c98700 1 -- 172.21.15.66:0/3870436331 learned_addr learned my addr 172.21.15.66:0/3870436331 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.081+0000 7fef71c98700 1 -- 172.21.15.66:0/3870436331 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c109f70 con 0x7fef6c108d70 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.082+0000 7fef71c98700 1 --2- 172.21.15.66:0/3870436331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c109190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef680098a0 tx=0x7fef68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5188c6988f2acbd0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.082+0000 7fef70c96700 1 -- 172.21.15.66:0/3870436331 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef68004160 con 0x7fef6c108d70 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.082+0000 7fef70c96700 1 -- 172.21.15.66:0/3870436331 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef680042c0 con 0x7fef6c108d70 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.082+0000 7fef70c96700 1 -- 172.21.15.66:0/3870436331 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef680118e0 con 0x7fef6c108d70 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 -- 172.21.15.66:0/3870436331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 msgr2=0x7fef6c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 --2- 172.21.15.66:0/3870436331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c109190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef680098a0 tx=0x7fef68009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 -- 172.21.15.66:0/3870436331 shutdown_connections 2023-12-11T19:24:16.314 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 --2- 172.21.15.66:0/3870436331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c109190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 -- 172.21.15.66:0/3870436331 >> 172.21.15.66:0/3870436331 conn(0x7fef6c07c180 msgr2=0x7fef6c07c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 -- 172.21.15.66:0/3870436331 shutdown_connections 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.083+0000 7fef73efc700 1 -- 172.21.15.66:0/3870436331 wait complete. 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.084+0000 7fef73efc700 1 Processor -- start 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.084+0000 7fef73efc700 1 -- start start 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.084+0000 7fef73efc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.084+0000 7fef73efc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c19d050 con 0x7fef6c108d70 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef71c98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef71c98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39966/0 (socket says 172.21.15.66:39966) 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef71c98700 1 -- 172.21.15.66:0/831044685 learned_addr learned my addr 172.21.15.66:0/831044685 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef71c98700 1 -- 172.21.15.66:0/831044685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009550 con 0x7fef6c108d70 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef71c98700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c19cb10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef68003f70 tx=0x7fef680119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef680037d0 con 0x7fef6c108d70 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef6801b440 con 0x7fef6c108d70 2023-12-11T19:24:16.315 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef680193d0 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.085+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c19d250 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.086+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c19d6f0 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.087+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fef6801b860 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.087+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c10d1a0 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.087+0000 7fef62ffd700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef58038870 0x7fef5803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.087+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef6804c810 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.087+0000 7fef71497700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef58038870 0x7fef5803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.088+0000 7fef71497700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef58038870 0x7fef5803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.092+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fef68018920 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.326+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fef6c19ffe0 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.326+0000 7fef62ffd700 1 -- 172.21.15.66:0/831044685 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fef6801f0b0 con 0x7fef6c108d70 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef58038870 msgr2=0x7fef5803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef58038870 0x7fef5803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:16.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 msgr2=0x7fef6c19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c19cb10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef68003f70 tx=0x7fef680119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 shutdown_connections 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef58038870 0x7fef5803ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 --2- 172.21.15.66:0/831044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef6c108d70 0x7fef6c19cb10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 >> 172.21.15.66:0/831044685 conn(0x7fef6c07c180 msgr2=0x7fef6c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 shutdown_connections 2023-12-11T19:24:16.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:15.329+0000 7fef73efc700 1 -- 172.21.15.66:0/831044685 wait complete. 2023-12-11T19:24:16.333 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T19:24:16.333 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-12-11T19:24:16.350 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T19:24:16.350 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-12-11T19:24:16.350 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-12-11T19:24:16.350 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-11T19:24:16.352 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph Dashboard is now available at: 2023-12-11T19:24:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: URL: https://smithi066.front.sepia.ceph.com:8443/ 2023-12-11T19:24:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: User: admin 2023-12-11T19:24:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: Password: mpm15am9rh 2023-12-11T19:24:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:16.353 INFO:teuthology.orchestra.run.smithi066.stdout:Saving cluster configuration to /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config directory 2023-12-11T19:24:16.353 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling autotune for osd_memory_target 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.889+0000 7fd1a9d92700 1 Processor -- start 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.890+0000 7fd1a9d92700 1 -- start start 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.890+0000 7fd1a9d92700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.890+0000 7fd1a9d92700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a407b8b0 con 0x7fd1a407ce80 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.891+0000 7fd1a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.891+0000 7fd1a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a407b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39972/0 (socket says 172.21.15.66:39972) 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.891+0000 7fd1a37fe700 1 -- 172.21.15.66:0/3891876095 learned_addr learned my addr 172.21.15.66:0/3891876095 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:17.538 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.891+0000 7fd1a37fe700 1 -- 172.21.15.66:0/3891876095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a407b9f0 con 0x7fd1a407ce80 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.891+0000 7fd1a37fe700 1 --2- 172.21.15.66:0/3891876095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a407b2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd1940098a0 tx=0x7fd194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a599dae79994337 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.891+0000 7fd1a27fc700 1 -- 172.21.15.66:0/3891876095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd194004160 con 0x7fd1a407ce80 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a27fc700 1 -- 172.21.15.66:0/3891876095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1940042c0 con 0x7fd1a407ce80 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a27fc700 1 -- 172.21.15.66:0/3891876095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1940118e0 con 0x7fd1a407ce80 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3891876095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 msgr2=0x7fd1a407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 --2- 172.21.15.66:0/3891876095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a407b2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd1940098a0 tx=0x7fd194009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3891876095 shutdown_connections 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 --2- 172.21.15.66:0/3891876095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a407b2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3891876095 >> 172.21.15.66:0/3891876095 conn(0x7fd1a41020c0 msgr2=0x7fd1a4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3891876095 shutdown_connections 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.892+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3891876095 wait complete. 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.893+0000 7fd1a9d92700 1 Processor -- start 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a9d92700 1 -- start start 2023-12-11T19:24:17.539 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a9d92700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a9d92700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419cfb0 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39988/0 (socket says 172.21.15.66:39988) 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a37fe700 1 -- 172.21.15.66:0/3041664691 learned_addr learned my addr 172.21.15.66:0/3041664691 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.894+0000 7fd1a37fe700 1 -- 172.21.15.66:0/3041664691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194009550 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.895+0000 7fd1a37fe700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a419ca70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd194003f70 tx=0x7fd1940119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.895+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1940036a0 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.895+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd19401b440 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.895+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a419d1b0 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.895+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd194023400 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.895+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a419d650 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.896+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fd194003800 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.896+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a410cf20 con 0x7fd1a407ce80 2023-12-11T19:24:17.540 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.897+0000 7fd1a0ff9700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd18c0388c0 0x7fd18c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.897+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd19404c960 con 0x7fd1a407ce80 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.897+0000 7fd1a2ffd700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd18c0388c0 0x7fd18c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.897+0000 7fd1a2ffd700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd18c0388c0 0x7fd18c03ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd198006fd0 tx=0x7fd198006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.902+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd19400bbd0 con 0x7fd1a407ce80 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:16.917+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fd194003ab0 con 0x7fd1a407ce80 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.113+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fd1a419fee0 con 0x7fd1a407ce80 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.113+0000 7fd1a0ff9700 1 -- 172.21.15.66:0/3041664691 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fd19402b350 con 0x7fd1a407ce80 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd18c0388c0 msgr2=0x7fd18c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd18c0388c0 0x7fd18c03ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd198006fd0 tx=0x7fd198006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 msgr2=0x7fd1a419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a419ca70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd194003f70 tx=0x7fd1940119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 shutdown_connections 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd18c0388c0 0x7fd18c03ad80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:17.541 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 --2- 172.21.15.66:0/3041664691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a407ce80 0x7fd1a419ca70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:17.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 >> 172.21.15.66:0/3041664691 conn(0x7fd1a41020c0 msgr2=0x7fd1a4102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:17.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 shutdown_connections 2023-12-11T19:24:17.542 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:17.116+0000 7fd1a9d92700 1 -- 172.21.15.66:0/3041664691 wait complete. 2023-12-11T19:24:18.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:17 smithi066 ceph-mon[97920]: mgrmap e13: x(active, since 6s) 2023-12-11T19:24:18.925 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.080+0000 7fe5b2533700 1 Processor -- start 2023-12-11T19:24:18.925 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.080+0000 7fe5b2533700 1 -- start start 2023-12-11T19:24:18.925 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.081+0000 7fe5b2533700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.081+0000 7fe5b2533700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac07b890 con 0x7fe5ac07ce60 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.081+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.081+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac07b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39992/0 (socket says 172.21.15.66:39992) 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.081+0000 7fe5abfff700 1 -- 172.21.15.66:0/3464309738 learned_addr learned my addr 172.21.15.66:0/3464309738 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.081+0000 7fe5abfff700 1 -- 172.21.15.66:0/3464309738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac07b9d0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.082+0000 7fe5abfff700 1 --2- 172.21.15.66:0/3464309738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac07b2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0098a0 tx=0x7fe59c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cded294ce2b3bafa server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.082+0000 7fe5aaffd700 1 -- 172.21.15.66:0/3464309738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe59c004160 con 0x7fe5ac07ce60 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.082+0000 7fe5aaffd700 1 -- 172.21.15.66:0/3464309738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe59c0042c0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.082+0000 7fe5aaffd700 1 -- 172.21.15.66:0/3464309738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe59c0118e0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 -- 172.21.15.66:0/3464309738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 msgr2=0x7fe5ac07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:18.926 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 --2- 172.21.15.66:0/3464309738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac07b2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0098a0 tx=0x7fe59c009bb0 comp rx=0 tx=0).stop 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 -- 172.21.15.66:0/3464309738 shutdown_connections 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 --2- 172.21.15.66:0/3464309738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac07b2c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 -- 172.21.15.66:0/3464309738 >> 172.21.15.66:0/3464309738 conn(0x7fe5ac1020c0 msgr2=0x7fe5ac104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 -- 172.21.15.66:0/3464309738 shutdown_connections 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.083+0000 7fe5b2533700 1 -- 172.21.15.66:0/3464309738 wait complete. 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.084+0000 7fe5b2533700 1 Processor -- start 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.085+0000 7fe5b2533700 1 -- start start 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.085+0000 7fe5b2533700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.085+0000 7fe5b2533700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac19cf40 con 0x7fe5ac07ce60 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.085+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.085+0000 7fe5abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39994/0 (socket says 172.21.15.66:39994) 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.085+0000 7fe5abfff700 1 -- 172.21.15.66:0/925138781 learned_addr learned my addr 172.21.15.66:0/925138781 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:18.927 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5abfff700 1 -- 172.21.15.66:0/925138781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c009550 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5abfff700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac19ca00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe59c003f70 tx=0x7fe59c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe59c0036a0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe59c01b440 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe59c023400 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac19d140 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.086+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac19d5e0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.087+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fe59c01b5b0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.087+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac10cf40 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.088+0000 7fe5a97fa700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe594038910 0x7fe59403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.088+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe59c04c9e0 con 0x7fe5ac07ce60 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.088+0000 7fe5ab7fe700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe594038910 0x7fe59403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:18.928 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.089+0000 7fe5ab7fe700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe594038910 0x7fe59403add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0006fd0 tx=0x7fe5a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.093+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe59c019840 con 0x7fe5ac07ce60 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.419+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe5ac19fe70 con 0x7fe5ac07ce60 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.421+0000 7fe5a97fa700 1 -- 172.21.15.66:0/925138781 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe59c019840 con 0x7fe5ac07ce60 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe594038910 msgr2=0x7fe59403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe594038910 0x7fe59403add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0006fd0 tx=0x7fe5a0006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 msgr2=0x7fe5ac19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac19ca00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe59c003f70 tx=0x7fe59c0119a0 comp rx=0 tx=0).stop 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 shutdown_connections 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe594038910 0x7fe59403add0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 --2- 172.21.15.66:0/925138781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5ac07ce60 0x7fe5ac19ca00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 >> 172.21.15.66:0/925138781 conn(0x7fe5ac1020c0 msgr2=0x7fe5ac102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 shutdown_connections 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-12-11T19:24:18.424+0000 7fe5b2533700 1 -- 172.21.15.66:0/925138781 wait complete. 2023-12-11T19:24:18.929 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout:Or, if you are only running a single cluster on this host: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: ceph telemetry on 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout:For more information see: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.930 INFO:teuthology.orchestra.run.smithi066.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-11T19:24:18.931 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:18.931 INFO:teuthology.orchestra.run.smithi066.stdout:Bootstrap complete. 2023-12-11T19:24:18.987 INFO:tasks.cephadm:Fetching config... 2023-12-11T19:24:18.987 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:24:18.988 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-11T19:24:19.032 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-11T19:24:19.032 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:24:19.032 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-11T19:24:19.120 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-11T19:24:19.120 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:24:19.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-11T19:24:19.233 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-11T19:24:19.233 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:24:19.233 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-11T19:24:19.319 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-11T19:24:19.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-11T19:24:19.466 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:24:19.489 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:19 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/925138781' entity='client.admin' 2023-12-11T19:24:19.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-11T19:24:19.602 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:24:19.632 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-11T19:24:19.738 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxR6mPH5JHKzczgZ9SBJYr/q5OmHD7H1nYZ0iJuSIc8bPXszQSWnP6wtmIh5LOfw80BKgw1r4kywimJvXHryHVQE6Fcwink21VoG8GTv5Ng+p2KDnCmWlct3Hx+gj8oZGiitcwWbp4f8t14nUegWQMS1KMZlsb5tz0PyUoJHHC+My1BfaU0g9ZgMZnX3TzQInlbTWN268OpsOLVZVv6enypkMQxDwbIfrrKtcVtBgR70ZQntH9iCddH1YaVsuSaCex5ZIo1pCJfkvxD9G53T7SSA/gIUycDjTry3QDXCVBTnVuY/5EUCdUkrLcRoBk2o71Q5ICvRPNZgTCk6KlbiM2llZf6B0/8hvJWUkSgABQSLX+j/pLn5LTPsANj5NfNPT3tZLR1D0X2LlQstLft9fyJkJps1mYNuD6xcUJcSkWdBKRVu6ibuk7Gwr1wZKleJu7tFJi4IsvOMkYKxfilP+eLSDwlVEfXICnAFJ4Ootgv5uW3XgVALMw3ran5/597es= ceph-976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:24:19.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-11T19:24:20.598 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.392+0000 7ff0bd85a700 1 -- 172.21.15.66:0/1348058284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 msgr2=0x7ff0b8101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.392+0000 7ff0bd85a700 1 --2- 172.21.15.66:0/1348058284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8101340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009910 tx=0x7ff0a8009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.392+0000 7ff0bd85a700 1 -- 172.21.15.66:0/1348058284 shutdown_connections 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.392+0000 7ff0bd85a700 1 --2- 172.21.15.66:0/1348058284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8101340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.392+0000 7ff0bd85a700 1 -- 172.21.15.66:0/1348058284 >> 172.21.15.66:0/1348058284 conn(0x7ff0b80fc4f0 msgr2=0x7ff0b80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.393+0000 7ff0bd85a700 1 -- 172.21.15.66:0/1348058284 shutdown_connections 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.393+0000 7ff0bd85a700 1 -- 172.21.15.66:0/1348058284 wait complete. 2023-12-11T19:24:22.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.394+0000 7ff0bd85a700 1 Processor -- start 2023-12-11T19:24:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.394+0000 7ff0bd85a700 1 -- start start 2023-12-11T19:24:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.394+0000 7ff0bd85a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:22.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.394+0000 7ff0bd85a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b81989d0 con 0x7ff0b8100f00 2023-12-11T19:24:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49630/0 (socket says 172.21.15.66:49630) 2023-12-11T19:24:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0b6ffd700 1 -- 172.21.15.66:0/3239034483 learned_addr learned my addr 172.21.15.66:0/3239034483 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0b6ffd700 1 -- 172.21.15.66:0/3239034483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80095f0 con 0x7ff0b8100f00 2023-12-11T19:24:22.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0b6ffd700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8198490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80098e0 tx=0x7ff0a8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0a800ba90 con 0x7ff0b8100f00 2023-12-11T19:24:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0a800bbf0 con 0x7ff0b8100f00 2023-12-11T19:24:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.395+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b8198c30 con 0x7ff0b8100f00 2023-12-11T19:24:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.396+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0a800fe70 con 0x7ff0b8100f00 2023-12-11T19:24:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.396+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b819b900 con 0x7ff0b8100f00 2023-12-11T19:24:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.397+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7ff0a800f460 con 0x7ff0b8100f00 2023-12-11T19:24:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.397+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8191e20 con 0x7ff0b8100f00 2023-12-11T19:24:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.397+0000 7ff0bc858700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff0a0038910 0x7ff0a003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.397+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff0a804ca70 con 0x7ff0b8100f00 2023-12-11T19:24:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.398+0000 7ff0b67fc700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff0a0038910 0x7ff0a003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.398+0000 7ff0b67fc700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff0a0038910 0x7ff0a003add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.403+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff0a802ab50 con 0x7ff0b8100f00 2023-12-11T19:24:22.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.614+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff0b8106d00 con 0x7ff0b8100f00 2023-12-11T19:24:22.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.620+0000 7ff0bc858700 1 -- 172.21.15.66:0/3239034483 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff0a8027030 con 0x7ff0b8100f00 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.622+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff0a0038910 msgr2=0x7ff0a003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.622+0000 7ff0bd85a700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff0a0038910 0x7ff0a003add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.622+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 msgr2=0x7ff0b8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.622+0000 7ff0bd85a700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8198490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80098e0 tx=0x7ff0a8005810 comp rx=0 tx=0).stop 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.623+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 shutdown_connections 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.623+0000 7ff0bd85a700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff0a0038910 0x7ff0a003add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.623+0000 7ff0bd85a700 1 --2- 172.21.15.66:0/3239034483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0b8100f00 0x7ff0b8198490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.623+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 >> 172.21.15.66:0/3239034483 conn(0x7ff0b80fc4f0 msgr2=0x7ff0b80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:22.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.623+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 shutdown_connections 2023-12-11T19:24:22.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:22.623+0000 7ff0bd85a700 1 -- 172.21.15.66:0/3239034483 wait complete. 2023-12-11T19:24:22.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:22.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:22.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:24:22.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:22.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:22.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:23.108 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-11T19:24:23.108 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-11T19:24:23.898 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:23 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3239034483' entity='client.admin' 2023-12-11T19:24:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.435+0000 7f56f9165700 1 -- 172.21.15.66:0/343957731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 msgr2=0x7f56f4101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.435+0000 7f56f9165700 1 --2- 172.21.15.66:0/343957731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4101340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009910 tx=0x7f56e4009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.436+0000 7f56f9165700 1 -- 172.21.15.66:0/343957731 shutdown_connections 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.436+0000 7f56f9165700 1 --2- 172.21.15.66:0/343957731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4101340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.436+0000 7f56f9165700 1 -- 172.21.15.66:0/343957731 >> 172.21.15.66:0/343957731 conn(0x7f56f40fc4f0 msgr2=0x7f56f40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.436+0000 7f56f9165700 1 -- 172.21.15.66:0/343957731 shutdown_connections 2023-12-11T19:24:25.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.436+0000 7f56f9165700 1 -- 172.21.15.66:0/343957731 wait complete. 2023-12-11T19:24:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f9165700 1 Processor -- start 2023-12-11T19:24:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f9165700 1 -- start start 2023-12-11T19:24:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f9165700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f9165700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f4075780 con 0x7f56f4100f00 2023-12-11T19:24:25.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:25.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4075240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49638/0 (socket says 172.21.15.66:49638) 2023-12-11T19:24:25.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.437+0000 7f56f2d9d700 1 -- 172.21.15.66:0/1004145866 learned_addr learned my addr 172.21.15.66:0/1004145866 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:25.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56f2d9d700 1 -- 172.21.15.66:0/1004145866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40095f0 con 0x7f56f4100f00 2023-12-11T19:24:25.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56f2d9d700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4075240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56e40098e0 tx=0x7f56e4005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:25.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56e400ba90 con 0x7f56f4100f00 2023-12-11T19:24:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e400bbf0 con 0x7f56f4100f00 2023-12-11T19:24:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56e400fe70 con 0x7f56f4100f00 2023-12-11T19:24:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f40739c0 con 0x7f56f4100f00 2023-12-11T19:24:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.438+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f4073ec0 con 0x7f56f4100f00 2023-12-11T19:24:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.439+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f56e400f460 con 0x7f56f4100f00 2023-12-11T19:24:25.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.439+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f4107350 con 0x7f56f4100f00 2023-12-11T19:24:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.440+0000 7f56dbfff700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56dc038910 0x7f56dc03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.440+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f56e404ca90 con 0x7f56f4100f00 2023-12-11T19:24:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.440+0000 7f56f259c700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56dc038910 0x7f56dc03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:25.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.441+0000 7f56f259c700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56dc038910 0x7f56dc03add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56e8006fd0 tx=0x7f56e8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:25.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.445+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f56e4015350 con 0x7f56f4100f00 2023-12-11T19:24:25.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.660+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f56f4062ca0 con 0x7f56dc038910 2023-12-11T19:24:25.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.663+0000 7f56dbfff700 1 -- 172.21.15.66:0/1004145866 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f56f4062ca0 con 0x7f56dc038910 2023-12-11T19:24:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56dc038910 msgr2=0x7f56dc03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56dc038910 0x7f56dc03add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56e8006fd0 tx=0x7f56e8006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 msgr2=0x7f56f4075240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4075240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56e40098e0 tx=0x7f56e4005810 comp rx=0 tx=0).stop 2023-12-11T19:24:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 shutdown_connections 2023-12-11T19:24:25.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56dc038910 0x7f56dc03add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 --2- 172.21.15.66:0/1004145866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56f4100f00 0x7f56f4075240 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 >> 172.21.15.66:0/1004145866 conn(0x7f56f40fc4f0 msgr2=0x7f56f40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 shutdown_connections 2023-12-11T19:24:25.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:25.666+0000 7f56f9165700 1 -- 172.21.15.66:0/1004145866 wait complete. 2023-12-11T19:24:26.195 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi070 2023-12-11T19:24:26.195 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:24:26.195 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/etc/ceph/ceph.conf 2023-12-11T19:24:26.243 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:24:26.243 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:24:26.329 INFO:tasks.cephadm:Adding host smithi070 to orchestrator... 2023-12-11T19:24:26.329 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch host add smithi070 2023-12-11T19:24:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:26 smithi066 ceph-mon[97920]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:24:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:24:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:26 smithi066 ceph-mon[97920]: Updating smithi066:/etc/ceph/ceph.conf 2023-12-11T19:24:27.149 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:27.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:27 smithi066 ceph-mon[97920]: Updating smithi066:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:24:27.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:27 smithi066 ceph-mon[97920]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 -- 172.21.15.66:0/3095411889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 msgr2=0x7f92a4101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 --2- 172.21.15.66:0/3095411889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a4101520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009910 tx=0x7f92a0009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 -- 172.21.15.66:0/3095411889 shutdown_connections 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 --2- 172.21.15.66:0/3095411889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a4101520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 -- 172.21.15.66:0/3095411889 >> 172.21.15.66:0/3095411889 conn(0x7f92a40fc9d0 msgr2=0x7f92a40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 -- 172.21.15.66:0/3095411889 shutdown_connections 2023-12-11T19:24:28.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.659+0000 7f92abcf4700 1 -- 172.21.15.66:0/3095411889 wait complete. 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.660+0000 7f92abcf4700 1 Processor -- start 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.660+0000 7f92abcf4700 1 -- start start 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.661+0000 7f92abcf4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.661+0000 7f92abcf4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a419eb70 con 0x7f92a4101140 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.661+0000 7f92a9a90700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.661+0000 7f92a9a90700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49658/0 (socket says 172.21.15.66:49658) 2023-12-11T19:24:28.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.661+0000 7f92a9a90700 1 -- 172.21.15.66:0/2583360088 learned_addr learned my addr 172.21.15.66:0/2583360088 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:28.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f92a9a90700 1 -- 172.21.15.66:0/2583360088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a00095f0 con 0x7f92a4101140 2023-12-11T19:24:28.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f92a9a90700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a419e630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92a00098e0 tx=0x7f92a0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:28.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92a001d070 con 0x7f92a4101140 2023-12-11T19:24:28.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92a0009e60 con 0x7f92a4101140 2023-12-11T19:24:28.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a419ed70 con 0x7f92a4101140 2023-12-11T19:24:28.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92a0022b30 con 0x7f92a4101140 2023-12-11T19:24:28.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.662+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a419f190 con 0x7f92a4101140 2023-12-11T19:24:28.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.663+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f92a0022470 con 0x7f92a4101140 2023-12-11T19:24:28.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.663+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a4102620 con 0x7f92a4101140 2023-12-11T19:24:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.664+0000 7f929affd700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9290038910 0x7f929003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.664+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f92a004cdf0 con 0x7f92a4101140 2023-12-11T19:24:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.664+0000 7f92a928f700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9290038910 0x7f929003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.665+0000 7f92a928f700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9290038910 0x7f929003add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:28.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.669+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f92a002ba00 con 0x7f92a4101140 2023-12-11T19:24:28.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:28.883+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi070", "target": ["mon-mgr", ""]}) v1 -- 0x7f92a410b0e0 con 0x7f9290038910 2023-12-11T19:24:29.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:28 smithi066 ceph-mon[97920]: Updating smithi066:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:24:29.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:28 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:29.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:28 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:29.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:28 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:30.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:29 smithi066 ceph-mon[97920]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:30 smithi066 ceph-mon[97920]: Deploying cephadm binary to smithi070 2023-12-11T19:24:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:31 smithi066 ceph-mon[97920]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:33.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.742+0000 7f929affd700 1 -- 172.21.15.66:0/2583360088 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f92a410b0e0 con 0x7f9290038910 2023-12-11T19:24:33.743 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi070' with addr '172.21.15.70' 2023-12-11T19:24:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9290038910 msgr2=0x7f929003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9290038910 0x7f929003add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 msgr2=0x7f92a419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a419e630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92a00098e0 tx=0x7f92a0005710 comp rx=0 tx=0).stop 2023-12-11T19:24:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 shutdown_connections 2023-12-11T19:24:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9290038910 0x7f929003add0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 --2- 172.21.15.66:0/2583360088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92a4101140 0x7f92a419e630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 >> 172.21.15.66:0/2583360088 conn(0x7f92a40fc9d0 msgr2=0x7f92a4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 shutdown_connections 2023-12-11T19:24:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:33.745+0000 7f92abcf4700 1 -- 172.21.15.66:0/2583360088 wait complete. 2023-12-11T19:24:34.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:33 smithi066 ceph-mon[97920]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:34.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:34.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:34.160 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch host ls --format=json 2023-12-11T19:24:34.959 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:35.252 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:34 smithi066 ceph-mon[97920]: Added host smithi070 2023-12-11T19:24:35.252 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:36.212 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:35 smithi066 ceph-mon[97920]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:36.212 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:36.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 -- 172.21.15.66:0/2374460762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 msgr2=0x7fb430101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:36.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 --2- 172.21.15.66:0/2374460762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb430101520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb420009910 tx=0x7fb420009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:36.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 -- 172.21.15.66:0/2374460762 shutdown_connections 2023-12-11T19:24:36.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 --2- 172.21.15.66:0/2374460762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb430101520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:36.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 -- 172.21.15.66:0/2374460762 >> 172.21.15.66:0/2374460762 conn(0x7fb4300fc9d0 msgr2=0x7fb4300fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:36.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 -- 172.21.15.66:0/2374460762 shutdown_connections 2023-12-11T19:24:36.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.473+0000 7fb43580d700 1 -- 172.21.15.66:0/2374460762 wait complete. 2023-12-11T19:24:36.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.474+0000 7fb43580d700 1 Processor -- start 2023-12-11T19:24:36.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.475+0000 7fb43580d700 1 -- start start 2023-12-11T19:24:36.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.475+0000 7fb43580d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb4300738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:36.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.475+0000 7fb43580d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430075790 con 0x7fb430101140 2023-12-11T19:24:36.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.476+0000 7fb42effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb4300738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:36.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.476+0000 7fb42effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb4300738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46564/0 (socket says 172.21.15.66:46564) 2023-12-11T19:24:36.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.476+0000 7fb42effd700 1 -- 172.21.15.66:0/3862927018 learned_addr learned my addr 172.21.15.66:0/3862927018 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:36.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.476+0000 7fb42effd700 1 -- 172.21.15.66:0/3862927018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4200095f0 con 0x7fb430101140 2023-12-11T19:24:36.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.476+0000 7fb42effd700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb4300738d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb420005660 tx=0x7fb420005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:36.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.476+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb42001d070 con 0x7fb430101140 2023-12-11T19:24:36.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.477+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb430073e10 con 0x7fb430101140 2023-12-11T19:24:36.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.477+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb42000b750 con 0x7fb430101140 2023-12-11T19:24:36.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.477+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4200037d0 con 0x7fb430101140 2023-12-11T19:24:36.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.477+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430074230 con 0x7fb430101140 2023-12-11T19:24:36.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.478+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fb420004390 con 0x7fb430101140 2023-12-11T19:24:36.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.478+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb430102620 con 0x7fb430101140 2023-12-11T19:24:36.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.478+0000 7fb43480b700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb4180388c0 0x7fb41803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:36.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.478+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb42004c550 con 0x7fb430101140 2023-12-11T19:24:36.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.479+0000 7fb42e7fc700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb4180388c0 0x7fb41803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:36.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.479+0000 7fb42e7fc700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb4180388c0 0x7fb41803ad80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:36.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.484+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb42000b8c0 con 0x7fb430101140 2023-12-11T19:24:36.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.699+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb430074470 con 0x7fb4180388c0 2023-12-11T19:24:36.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.700+0000 7fb43480b700 1 -- 172.21.15.66:0/3862927018 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fb430074470 con 0x7fb4180388c0 2023-12-11T19:24:36.701 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:36.701 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}] 2023-12-11T19:24:36.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.702+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb4180388c0 msgr2=0x7fb41803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.702+0000 7fb43580d700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb4180388c0 0x7fb41803ad80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.702+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 msgr2=0x7fb4300738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.702+0000 7fb43580d700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb4300738d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb420005660 tx=0x7fb420005740 comp rx=0 tx=0).stop 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.703+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 shutdown_connections 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.703+0000 7fb43580d700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb4180388c0 0x7fb41803ad80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.703+0000 7fb43580d700 1 --2- 172.21.15.66:0/3862927018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb430101140 0x7fb4300738d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.703+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 >> 172.21.15.66:0/3862927018 conn(0x7fb4300fc9d0 msgr2=0x7fb430102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.703+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 shutdown_connections 2023-12-11T19:24:36.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:36.703+0000 7fb43580d700 1 -- 172.21.15.66:0/3862927018 wait complete. 2023-12-11T19:24:37.055 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2023-12-11T19:24:37.055 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:24:37.055 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2023-12-11T19:24:37.103 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:24:37.103 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:24:37.189 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2023-12-11T19:24:37.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch host add smithi167 2023-12-11T19:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:38.235 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:37 smithi066 ceph-mon[97920]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:38.235 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:37 smithi066 ceph-mon[97920]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:24:38.236 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.490+0000 7fef21d98700 1 -- 172.21.15.66:0/3771430661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 msgr2=0x7fef1c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.490+0000 7fef21d98700 1 --2- 172.21.15.66:0/3771430661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c101520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009910 tx=0x7fef0c009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.490+0000 7fef21d98700 1 -- 172.21.15.66:0/3771430661 shutdown_connections 2023-12-11T19:24:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.490+0000 7fef21d98700 1 --2- 172.21.15.66:0/3771430661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c101520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.490+0000 7fef21d98700 1 -- 172.21.15.66:0/3771430661 >> 172.21.15.66:0/3771430661 conn(0x7fef1c0fc9d0 msgr2=0x7fef1c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:39.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.491+0000 7fef21d98700 1 -- 172.21.15.66:0/3771430661 shutdown_connections 2023-12-11T19:24:39.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.491+0000 7fef21d98700 1 -- 172.21.15.66:0/3771430661 wait complete. 2023-12-11T19:24:39.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.492+0000 7fef21d98700 1 Processor -- start 2023-12-11T19:24:39.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.492+0000 7fef21d98700 1 -- start start 2023-12-11T19:24:39.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.492+0000 7fef21d98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:39.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.492+0000 7fef21d98700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c19ec10 con 0x7fef1c101140 2023-12-11T19:24:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef1b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef1b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46580/0 (socket says 172.21.15.66:46580) 2023-12-11T19:24:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef1b7fe700 1 -- 172.21.15.66:0/1018586229 learned_addr learned my addr 172.21.15.66:0/1018586229 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef1b7fe700 1 -- 172.21.15.66:0/1018586229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0095f0 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef1b7fe700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c19e6d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005660 tx=0x7fef0c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef0c01d070 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef0c00b750 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c19edb0 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.493+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef0c0037d0 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.494+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c19f1d0 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.495+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fef0c004390 con 0x7fef1c101140 2023-12-11T19:24:39.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.495+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c102620 con 0x7fef1c101140 2023-12-11T19:24:39.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.495+0000 7fef18ff9700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef04038910 0x7fef0403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:39.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.495+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef0c04c590 con 0x7fef1c101140 2023-12-11T19:24:39.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.495+0000 7fef1affd700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef04038910 0x7fef0403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:39.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.496+0000 7fef1affd700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef04038910 0x7fef0403add0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef10006fd0 tx=0x7fef10006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:39.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.500+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fef0c003a40 con 0x7fef1c101140 2023-12-11T19:24:39.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:39.715+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) v1 -- 0x7fef1c10b0e0 con 0x7fef04038910 2023-12-11T19:24:40.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:39 smithi066 ceph-mon[97920]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:41.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:24:41.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:24:41.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:24:42.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:41 smithi066 ceph-mon[97920]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:42.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:41 smithi066 ceph-mon[97920]: Deploying cephadm binary to smithi167 2023-12-11T19:24:42.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:41 smithi066 ceph-mon[97920]: Updating smithi070:/etc/ceph/ceph.conf 2023-12-11T19:24:43.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:42 smithi066 ceph-mon[97920]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:24:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:43 smithi066 ceph-mon[97920]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:44.625 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi167' with addr '172.21.15.167' 2023-12-11T19:24:44.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.623+0000 7fef18ff9700 1 -- 172.21.15.66:0/1018586229 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fef1c10b0e0 con 0x7fef04038910 2023-12-11T19:24:44.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef04038910 msgr2=0x7fef0403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:44.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef04038910 0x7fef0403add0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef10006fd0 tx=0x7fef10006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:44.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 msgr2=0x7fef1c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:44.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c19e6d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005660 tx=0x7fef0c005740 comp rx=0 tx=0).stop 2023-12-11T19:24:44.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 shutdown_connections 2023-12-11T19:24:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fef04038910 0x7fef0403add0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 --2- 172.21.15.66:0/1018586229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef1c101140 0x7fef1c19e6d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.626+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 >> 172.21.15.66:0/1018586229 conn(0x7fef1c0fc9d0 msgr2=0x7fef1c104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.627+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 shutdown_connections 2023-12-11T19:24:44.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:44.627+0000 7fef21d98700 1 -- 172.21.15.66:0/1018586229 wait complete. 2023-12-11T19:24:45.087 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch host ls --format=json 2023-12-11T19:24:45.895 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: Added host smithi167 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 -- 172.21.15.66:0/2136633257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 msgr2=0x7f5e58101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 --2- 172.21.15.66:0/2136633257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e58101520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009910 tx=0x7f5e54009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 -- 172.21.15.66:0/2136633257 shutdown_connections 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 --2- 172.21.15.66:0/2136633257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e58101520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 -- 172.21.15.66:0/2136633257 >> 172.21.15.66:0/2136633257 conn(0x7f5e580fc9d0 msgr2=0x7f5e580fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 -- 172.21.15.66:0/2136633257 shutdown_connections 2023-12-11T19:24:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.298+0000 7f5e6068a700 1 -- 172.21.15.66:0/2136633257 wait complete. 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.299+0000 7f5e6068a700 1 Processor -- start 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.300+0000 7f5e6068a700 1 -- start start 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.300+0000 7f5e6068a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.300+0000 7f5e6068a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5819eb70 con 0x7f5e58101140 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.300+0000 7f5e5e426700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.300+0000 7f5e5e426700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45572/0 (socket says 172.21.15.66:45572) 2023-12-11T19:24:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.300+0000 7f5e5e426700 1 -- 172.21.15.66:0/598870878 learned_addr learned my addr 172.21.15.66:0/598870878 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:47.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e5e426700 1 -- 172.21.15.66:0/598870878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540095f0 con 0x7f5e58101140 2023-12-11T19:24:47.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e5e426700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e540098e0 tx=0x7f5e54005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:47.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e5401d070 con 0x7f5e58101140 2023-12-11T19:24:47.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e54009e60 con 0x7f5e58101140 2023-12-11T19:24:47.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e54022b30 con 0x7f5e58101140 2023-12-11T19:24:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5819ed70 con 0x7f5e58101140 2023-12-11T19:24:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.301+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5819f190 con 0x7f5e58101140 2023-12-11T19:24:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.302+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e58102620 con 0x7f5e58101140 2023-12-11T19:24:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.302+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f5e54022470 con 0x7f5e58101140 2023-12-11T19:24:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.303+0000 7f5e4f7fe700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e44038910 0x7f5e4403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.303+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5e5404cd30 con 0x7f5e58101140 2023-12-11T19:24:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.303+0000 7f5e5dc25700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e44038910 0x7f5e4403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.304+0000 7f5e5dc25700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e44038910 0x7f5e4403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5e48006fd0 tx=0x7f5e48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.308+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e5402ba00 con 0x7f5e58101140 2023-12-11T19:24:47.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:47.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:47 smithi066 ceph-mon[97920]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:47.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:47.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.524+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e5810b0e0 con 0x7f5e44038910 2023-12-11T19:24:47.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.525+0000 7f5e4f7fe700 1 -- 172.21.15.66:0/598870878 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f5e5810b0e0 con 0x7f5e44038910 2023-12-11T19:24:47.526 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:24:47.526 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2023-12-11T19:24:47.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.527+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e44038910 msgr2=0x7f5e4403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.527+0000 7f5e6068a700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e44038910 0x7f5e4403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5e48006fd0 tx=0x7f5e48006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.527+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 msgr2=0x7f5e5819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.527+0000 7f5e6068a700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e540098e0 tx=0x7f5e54005710 comp rx=0 tx=0).stop 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.528+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 shutdown_connections 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.528+0000 7f5e6068a700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e44038910 0x7f5e4403add0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.528+0000 7f5e6068a700 1 --2- 172.21.15.66:0/598870878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e58101140 0x7f5e5819e630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.528+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 >> 172.21.15.66:0/598870878 conn(0x7f5e580fc9d0 msgr2=0x7f5e58073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.528+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 shutdown_connections 2023-12-11T19:24:47.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:47.528+0000 7f5e6068a700 1 -- 172.21.15.66:0/598870878 wait complete. 2023-12-11T19:24:48.092 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-11T19:24:48.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd crush tunables default 2023-12-11T19:24:48.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:48 smithi066 ceph-mon[97920]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:24:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:24:49.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:49 smithi066 ceph-mon[97920]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:49.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.349+0000 7f1dd2491700 1 -- 172.21.15.66:0/2201929326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 msgr2=0x7f1dcc107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.349+0000 7f1dd2491700 1 --2- 172.21.15.66:0/2201929326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc107360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009910 tx=0x7f1dbc009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.350+0000 7f1dd2491700 1 -- 172.21.15.66:0/2201929326 shutdown_connections 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.350+0000 7f1dd2491700 1 --2- 172.21.15.66:0/2201929326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc107360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.350+0000 7f1dd2491700 1 -- 172.21.15.66:0/2201929326 >> 172.21.15.66:0/2201929326 conn(0x7f1dcc075930 msgr2=0x7f1dcc075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.350+0000 7f1dd2491700 1 -- 172.21.15.66:0/2201929326 shutdown_connections 2023-12-11T19:24:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.350+0000 7f1dd2491700 1 -- 172.21.15.66:0/2201929326 wait complete. 2023-12-11T19:24:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.351+0000 7f1dd2491700 1 Processor -- start 2023-12-11T19:24:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.351+0000 7f1dd2491700 1 -- start start 2023-12-11T19:24:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.352+0000 7f1dd2491700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.352+0000 7f1dd2491700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc196c40 con 0x7f1dcc106f80 2023-12-11T19:24:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.352+0000 7f1dcbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.352+0000 7f1dcbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45582/0 (socket says 172.21.15.66:45582) 2023-12-11T19:24:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.352+0000 7f1dcbfff700 1 -- 172.21.15.66:0/3245414333 learned_addr learned my addr 172.21.15.66:0/3245414333 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:24:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dcbfff700 1 -- 172.21.15.66:0/3245414333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc0095f0 con 0x7f1dcc106f80 2023-12-11T19:24:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dcbfff700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc196700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0098e0 tx=0x7f1dbc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc01d070 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dbc009e60 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc022b30 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc196ea0 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.353+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc197320 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.355+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f1dbc022470 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.355+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc102630 con 0x7f1dcc106f80 2023-12-11T19:24:50.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.355+0000 7f1dc97fa700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1db4038910 0x7f1db403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.355+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1dbc04cdf0 con 0x7f1dcc106f80 2023-12-11T19:24:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.355+0000 7f1dcb7fe700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1db4038910 0x7f1db403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.356+0000 7f1dcb7fe700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1db4038910 0x7f1db403add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.360+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1dbc02ba00 con 0x7f1dcc106f80 2023-12-11T19:24:50.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.577+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1dcc10b080 con 0x7f1dcc106f80 2023-12-11T19:24:50.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.734+0000 7f1dc97fa700 1 -- 172.21.15.66:0/3245414333 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1dbc027070 con 0x7f1dcc106f80 2023-12-11T19:24:50.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1db4038910 msgr2=0x7f1db403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1db4038910 0x7f1db403add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 msgr2=0x7f1dcc196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc196700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0098e0 tx=0x7f1dbc005710 comp rx=0 tx=0).stop 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 shutdown_connections 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1db4038910 0x7f1db403add0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 --2- 172.21.15.66:0/3245414333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dcc106f80 0x7f1dcc196700 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 >> 172.21.15.66:0/3245414333 conn(0x7f1dcc075930 msgr2=0x7f1dcc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 shutdown_connections 2023-12-11T19:24:50.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:24:50.737+0000 7f1dd2491700 1 -- 172.21.15.66:0/3245414333 wait complete. 2023-12-11T19:24:50.740 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2023-12-11T19:24:51.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:50 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3245414333' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-11T19:24:51.182 INFO:tasks.cephadm:Adding mon.a on smithi066 2023-12-11T19:24:51.182 INFO:tasks.cephadm:Adding mon.b on smithi070 2023-12-11T19:24:51.182 INFO:tasks.cephadm:Adding mon.c on smithi167 2023-12-11T19:24:51.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch apply mon '3;smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c' 2023-12-11T19:24:52.002 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /etc/ceph/ceph.conf 2023-12-11T19:24:52.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:51 smithi066 ceph-mon[97920]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:52.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:51 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3245414333' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-11T19:24:52.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:51 smithi066 ceph-mon[97920]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:24:52.606 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /etc/ceph/ceph.conf 2023-12-11T19:24:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:24:53.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:24:53.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:24:53.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: Updating smithi167:/etc/ceph/ceph.conf 2023-12-11T19:24:53.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:53 smithi066 ceph-mon[97920]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:24:54.033 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 -- 172.21.15.167:0/4174868182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 msgr2=0x7f7298103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:54.033 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 --2- 172.21.15.167:0/4174868182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298103580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7294009910 tx=0x7f7294009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:54.033 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 -- 172.21.15.167:0/4174868182 shutdown_connections 2023-12-11T19:24:54.033 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 --2- 172.21.15.167:0/4174868182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298103580 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:54.034 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 -- 172.21.15.167:0/4174868182 >> 172.21.15.167:0/4174868182 conn(0x7f72980fe700 msgr2=0x7f7298100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:54.034 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 -- 172.21.15.167:0/4174868182 shutdown_connections 2023-12-11T19:24:54.034 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.031+0000 7f729fd20700 1 -- 172.21.15.167:0/4174868182 wait complete. 2023-12-11T19:24:54.034 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.033+0000 7f729fd20700 1 Processor -- start 2023-12-11T19:24:54.034 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.033+0000 7f729fd20700 1 -- start start 2023-12-11T19:24:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.033+0000 7f729fd20700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.033+0000 7f729fd20700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298198a40 con 0x7f7298103160 2023-12-11T19:24:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.034+0000 7f729dabc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.034+0000 7f729dabc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.167:55996/0 (socket says 172.21.15.167:55996) 2023-12-11T19:24:54.035 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.034+0000 7f729dabc700 1 -- 172.21.15.167:0/436858537 learned_addr learned my addr 172.21.15.167:0/436858537 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:24:54.036 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.034+0000 7f729dabc700 1 -- 172.21.15.167:0/436858537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940095f0 con 0x7f7298103160 2023-12-11T19:24:54.036 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.034+0000 7f729dabc700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298198500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7294005790 tx=0x7f72940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:54.036 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.035+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f729400b960 con 0x7f7298103160 2023-12-11T19:24:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.035+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298198c40 con 0x7f7298103160 2023-12-11T19:24:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.035+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f729400f460 con 0x7f7298103160 2023-12-11T19:24:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.035+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72940175d0 con 0x7f7298103160 2023-12-11T19:24:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.035+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72981990e0 con 0x7f7298103160 2023-12-11T19:24:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.036+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f729400f5d0 con 0x7f7298103160 2023-12-11T19:24:54.037 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.036+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7298191d30 con 0x7f7298103160 2023-12-11T19:24:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.036+0000 7f728effd700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7284038910 0x7f728403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.036+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f729404ca50 con 0x7f7298103160 2023-12-11T19:24:54.038 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.037+0000 7f729d2bb700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7284038910 0x7f728403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:54.039 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.037+0000 7f729d2bb700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7284038910 0x7f728403add0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:54.044 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.042+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f729402ab50 con 0x7f7298103160 2023-12-11T19:24:54.271 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.269+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7298061ad0 con 0x7f7284038910 2023-12-11T19:24:54.278 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.277+0000 7f728effd700 1 -- 172.21.15.167:0/436858537 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7298061ad0 con 0x7f7284038910 2023-12-11T19:24:54.279 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2023-12-11T19:24:54.281 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.279+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7284038910 msgr2=0x7f728403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:54.281 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.279+0000 7f729fd20700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7284038910 0x7f728403add0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:54.282 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.279+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 msgr2=0x7f7298198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:54.282 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.279+0000 7f729fd20700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298198500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7294005790 tx=0x7f72940047b0 comp rx=0 tx=0).stop 2023-12-11T19:24:54.282 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.280+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 shutdown_connections 2023-12-11T19:24:54.282 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.280+0000 7f729fd20700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7284038910 0x7f728403add0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:54.282 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.280+0000 7f729fd20700 1 --2- 172.21.15.167:0/436858537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7298103160 0x7f7298198500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:54.283 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.280+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 >> 172.21.15.167:0/436858537 conn(0x7f72980fe700 msgr2=0x7f72980ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:54.283 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.280+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 shutdown_connections 2023-12-11T19:24:54.283 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:54.280+0000 7f729fd20700 1 -- 172.21.15.167:0/436858537 wait complete. 2023-12-11T19:24:54.810 DEBUG:teuthology.orchestra.run.smithi070:mon.b> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.b.service 2023-12-11T19:24:54.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:54 smithi066 ceph-mon[97920]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:24:54.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:54 smithi066 ceph-mon[97920]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:24:54.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:54.815 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.c.service 2023-12-11T19:24:54.820 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-11T19:24:54.820 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-11T19:24:54.891 INFO:journalctl@ceph.mon.b.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:24:54.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:24:55.789 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c;count:3 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:24:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:24:56.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:55 smithi066 ceph-mon[97920]: Deploying daemon mon.c on smithi167 2023-12-11T19:24:58.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:57 smithi066 ceph-mon[97920]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 -- 172.21.15.167:0/1517905426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 msgr2=0x7fc520107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 --2- 172.21.15.167:0/1517905426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520107360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc510009910 tx=0x7fc510009c20 comp rx=0 tx=0).stop 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 -- 172.21.15.167:0/1517905426 shutdown_connections 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 --2- 172.21.15.167:0/1517905426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520107360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 -- 172.21.15.167:0/1517905426 >> 172.21.15.167:0/1517905426 conn(0x7fc520075930 msgr2=0x7fc520075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 -- 172.21.15.167:0/1517905426 shutdown_connections 2023-12-11T19:24:58.098 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.096+0000 7fc525680700 1 -- 172.21.15.167:0/1517905426 wait complete. 2023-12-11T19:24:58.099 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.097+0000 7fc525680700 1 Processor -- start 2023-12-11T19:24:58.099 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.097+0000 7fc525680700 1 -- start start 2023-12-11T19:24:58.099 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.098+0000 7fc525680700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:58.100 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.098+0000 7fc525680700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc520196c40 con 0x7fc520106f80 2023-12-11T19:24:58.100 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.098+0000 7fc51effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:58.100 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.098+0000 7fc51effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.167:56010/0 (socket says 172.21.15.167:56010) 2023-12-11T19:24:58.100 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.098+0000 7fc51effd700 1 -- 172.21.15.167:0/3690909876 learned_addr learned my addr 172.21.15.167:0/3690909876 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:24:58.100 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.099+0000 7fc51effd700 1 -- 172.21.15.167:0/3690909876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100095f0 con 0x7fc520106f80 2023-12-11T19:24:58.101 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.099+0000 7fc51effd700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520196700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098e0 tx=0x7fc510005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:58.101 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.099+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc51001d070 con 0x7fc520106f80 2023-12-11T19:24:58.101 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.099+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc510009e60 con 0x7fc520106f80 2023-12-11T19:24:58.101 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.099+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc510022b30 con 0x7fc520106f80 2023-12-11T19:24:58.101 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.099+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc520196ea0 con 0x7fc520106f80 2023-12-11T19:24:58.101 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.100+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc520197320 con 0x7fc520106f80 2023-12-11T19:24:58.103 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.101+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc520102630 con 0x7fc520106f80 2023-12-11T19:24:58.103 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.101+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fc510022470 con 0x7fc520106f80 2023-12-11T19:24:58.103 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.102+0000 7fc507fff700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc5080388c0 0x7fc50803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:24:58.103 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.102+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc51004ccb0 con 0x7fc520106f80 2023-12-11T19:24:58.103 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.102+0000 7fc51e7fc700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc5080388c0 0x7fc50803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:24:58.104 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.103+0000 7fc51e7fc700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc5080388c0 0x7fc50803ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc514006fd0 tx=0x7fc514006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:24:58.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.108+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc51002ba00 con 0x7fc520106f80 2023-12-11T19:24:58.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.416+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc52010b080 con 0x7fc520106f80 2023-12-11T19:24:58.418 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.417+0000 7fc507fff700 1 -- 172.21.15.167:0/3690909876 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fc510027020 con 0x7fc520106f80 2023-12-11T19:24:58.419 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:24:58.419 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","modified":"2023-12-11T19:23:04.843190Z","created":"2023-12-11T19:23:04.843190Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc5080388c0 msgr2=0x7fc50803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc5080388c0 0x7fc50803ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc514006fd0 tx=0x7fc514006e40 comp rx=0 tx=0).stop 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 msgr2=0x7fc520196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520196700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5100098e0 tx=0x7fc510005710 comp rx=0 tx=0).stop 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 shutdown_connections 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc5080388c0 0x7fc50803ad80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:58.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 --2- 172.21.15.167:0/3690909876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc520106f80 0x7fc520196700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:24:58.422 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 >> 172.21.15.167:0/3690909876 conn(0x7fc520075930 msgr2=0x7fc5200fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:24:58.422 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 shutdown_connections 2023-12-11T19:24:58.422 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:24:58.419+0000 7fc525680700 1 -- 172.21.15.167:0/3690909876 wait complete. 2023-12-11T19:24:58.422 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2023-12-11T19:24:59.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:58 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/3690909876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T19:24:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 systemd[1]: Starting Ceph mon.c for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:24:59.884 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 podman[102710]: 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 bash[102710]: 20afe80bde5553a7c061803ac7f9cb92d3d76ed362dc5c7ac872c0e3589e0122 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: set uid:gid to 167:167 (ceph:ceph) 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pidfile_write: ignore empty --pid-file 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: load: jerasure load: lrc 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: RocksDB version: 7.9.2 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Git sha 0 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: DB SUMMARY 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: DB Session ID: PNDUBEST6EIKMV5HPV73 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: CURRENT file: CURRENT 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: IDENTITY file: IDENTITY 2023-12-11T19:24:59.885 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.error_if_exists: 0 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.create_if_missing: 0 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.paranoid_checks: 1 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.env: 0x55699e072a60 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.fs: PosixFileSystem 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.info_log: 0x5569a0745780 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_file_opening_threads: 16 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.statistics: (nil) 2023-12-11T19:24:59.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.use_fsync: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_log_file_size: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.keep_log_file_num: 1000 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.recycle_log_file_num: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.allow_fallocate: 1 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.allow_mmap_reads: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.allow_mmap_writes: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.use_direct_reads: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.create_missing_column_families: 0 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.db_log_dir: 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.wal_dir: 2023-12-11T19:24:59.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.advise_random_on_open: 1 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.db_write_buffer_size: 0 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.write_buffer_manager: 0x5569a06fb360 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.rate_limiter: (nil) 2023-12-11T19:24:59.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.wal_recovery_mode: 2 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.enable_thread_tracking: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.enable_pipelined_write: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.unordered_write: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.row_cache: None 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.wal_filter: None 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.allow_ingest_behind: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.two_write_queues: 0 2023-12-11T19:24:59.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.manual_wal_flush: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.wal_compression: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.atomic_flush: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.log_readahead_size: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.best_efforts_recovery: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.allow_data_in_errors: 0 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.db_host_id: __hostname__ 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-11T19:24:59.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_background_jobs: 2 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_background_compactions: -1 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_subcompactions: 1 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_total_wal_size: 0 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_open_files: -1 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bytes_per_sync: 0 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-11T19:24:59.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_readahead_size: 0 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_background_flushes: -1 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Compression algorithms supported: 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kZSTD supported: 0 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kXpressCompression supported: 0 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kLZ4HCCompression supported: 1 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kZlibCompression supported: 1 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kSnappyCompression supported: 1 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kLZ4Compression supported: 1 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: kBZip2Compression supported: 0 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-11T19:24:59.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.merge_operator: 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_filter: None 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_filter_factory: None 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.sst_partitioner_factory: None 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5569a07458a0) 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks: 1 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_top_level_index_and_filter: 1 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_type: 0 2023-12-11T19:24:59.893 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_index_type: 0 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_shortening: 1 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: checksum: 4 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: no_block_cache: 0 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache: 0x5569a079b090 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_name: BinnedLRUCache 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_options: 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: capacity : 536870912 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_shard_bits : 4 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: strict_capacity_limit : 0 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: high_pri_pool_ratio: 0.000 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_compressed: (nil) 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: persistent_cache: (nil) 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size: 4096 2023-12-11T19:24:59.894 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size_deviation: 10 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_restart_interval: 16 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_block_restart_interval: 1 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: metadata_block_size: 4096 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: partition_filters: 0 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: use_delta_encoding: 1 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: filter_policy: bloomfilter 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: whole_key_filtering: 1 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: verify_compression: 0 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: read_amp_bytes_per_bit: 0 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: format_version: 5 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_index_compression: 1 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_align: 0 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: max_auto_readahead_size: 262144 2023-12-11T19:24:59.895 INFO:journalctl@ceph.mon.c.smithi167.stdout: prepopulate_block_cache: 0 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout: initial_auto_readahead_size: 8192 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.write_buffer_size: 33554432 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_write_buffer_number: 2 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression: NoCompression 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression: Disabled 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.prefix_extractor: nullptr 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.num_levels: 7 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-11T19:24:59.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.level: 32767 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.strategy: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-11T19:24:59.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.enabled: false 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.target_file_size_base: 67108864 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-11T19:24:59.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.arena_block_size: 1048576 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.disable_auto_compactions: 0 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-11T19:24:59.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.table_properties_collectors: 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.inplace_update_support: 0 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-11T19:24:59.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.bloom_locality: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.max_successive_merges: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.paranoid_file_checks: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.force_consistency_checks: 1 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.report_bg_io_stats: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.ttl: 2592000 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.enable_blob_files: false 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.min_blob_size: 0 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.blob_file_size: 268435456 2023-12-11T19:24:59.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.blob_file_starting_level: 0 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 62045e10-9638-4a97-a617-1668947e71d6 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322699852109, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322699853129, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702322699, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "62045e10-9638-4a97-a617-1668947e71d6", "db_session_id": "PNDUBEST6EIKMV5HPV73", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-11T19:24:59.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322699853340, "job": 1, "event": "recovery_finished"} 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5569a0836000 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: DB pointer 0x5569a0822000 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** DB Stats ** 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T19:24:59.903 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2023-12-11T19:24:59.904 INFO:journalctl@ceph.mon.c.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache BinnedLRUCache@0x5569a079b090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-11T19:24:59.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: using public_addr v2:172.21.15.167:0/0 -> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: starting mon.c rank -1 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(???) e0 preinit fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).mds e1 new map 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout: e1 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout: legacy client fscid: -1 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout: No filesystems configured 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-11T19:24:59.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:24:59.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:24:59.908 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:59.908 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:59.908 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:59.908 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:24:59.908 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-11T19:24:59.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-11T19:25:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:24:59 smithi066 ceph-mon[97920]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Deploying cephadm binary to smithi167 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi070:/etc/ceph/ceph.conf 2023-12-11T19:25:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Added host smithi167 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3245414333' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3245414333' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi167:/etc/ceph/ceph.conf 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c;count:3 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:00.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: Deploying daemon mon.c on smithi167 2023-12-11T19:25:00.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/3690909876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T19:25:00.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:00.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 ceph-mon[102750]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-11T19:25:00.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:24:59 smithi167 systemd[1]: Started Ceph mon.c for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:25:00.686 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:25:05.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:25:05.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: mon.a calling monitor election 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: mon.c calling monitor election 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: fsmap 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: mgrmap e13: x(active, since 54s) 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: overall HEALTH_OK 2023-12-11T19:25:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:05.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:25:05.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: mon.a calling monitor election 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: mon.c calling monitor election 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: fsmap 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: mgrmap e13: x(active, since 54s) 2023-12-11T19:25:05.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: overall HEALTH_OK 2023-12-11T19:25:05.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:05.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:05.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 -- 172.21.15.167:0/2021979985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 msgr2=0x7f08a4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:05.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 --2- 172.21.15.167:0/2021979985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08a4005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08b8100f40 tx=0x7f08b400feb0 comp rx=0 tx=0).stop 2023-12-11T19:25:05.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 -- 172.21.15.167:0/2021979985 shutdown_connections 2023-12-11T19:25:05.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 --2- 172.21.15.167:0/2021979985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08a4005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:05.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 -- 172.21.15.167:0/2021979985 >> 172.21.15.167:0/2021979985 conn(0x7f08b80fcb20 msgr2=0x7f08b80fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:05.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 -- 172.21.15.167:0/2021979985 shutdown_connections 2023-12-11T19:25:05.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.344+0000 7f08c03fc700 1 -- 172.21.15.167:0/2021979985 wait complete. 2023-12-11T19:25:05.347 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.346+0000 7f08c03fc700 1 Processor -- start 2023-12-11T19:25:05.348 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.346+0000 7f08c03fc700 1 -- start start 2023-12-11T19:25:05.349 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08c03fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08b819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:05.349 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08c03fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b819ed90 0x7f08b8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:05.349 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08c03fc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b819f2b0 con 0x7f08a40035c0 2023-12-11T19:25:05.349 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08c03fc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8198e90 con 0x7f08b819ed90 2023-12-11T19:25:05.349 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08be198700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08b819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:05.349 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08be198700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08b819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.167:38276/0 (socket says 172.21.15.167:38276) 2023-12-11T19:25:05.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.347+0000 7f08be198700 1 -- 172.21.15.167:0/1565671901 learned_addr learned my addr 172.21.15.167:0/1565671901 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:25:05.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.348+0000 7f08bd997700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b819ed90 0x7f08b8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:05.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.348+0000 7f08be198700 1 -- 172.21.15.167:0/1565671901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b819ed90 msgr2=0x7f08b8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.348+0000 7f08be198700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b819ed90 0x7f08b8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.348+0000 7f08be198700 1 -- 172.21.15.167:0/1565671901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b400f900 con 0x7f08a40035c0 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.348+0000 7f08be198700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08b819e850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08b40059d0 tx=0x7f08b4005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.348+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f08b4013b70 con 0x7f08a40035c0 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.349+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08b40044c0 con 0x7f08a40035c0 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.349+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b81990c0 con 0x7f08a40035c0 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.349+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f08b4003c50 con 0x7f08a40035c0 2023-12-11T19:25:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.349+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b8199610 con 0x7f08a40035c0 2023-12-11T19:25:05.352 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.350+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f08b4004030 con 0x7f08a40035c0 2023-12-11T19:25:05.352 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.350+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8199880 con 0x7f08a40035c0 2023-12-11T19:25:05.352 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.351+0000 7f08af7fe700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f089c038700 0x7f089c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:05.352 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.351+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f08b404df00 con 0x7f08a40035c0 2023-12-11T19:25:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.351+0000 7f08bd997700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f089c038700 0x7f089c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.352+0000 7f08bd997700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f089c038700 0x7f089c03abc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009850 tx=0x7f08a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:05.358 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.357+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f08b40349a0 con 0x7f08a40035c0 2023-12-11T19:25:05.638 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.636+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f08b8062c00 con 0x7f08a40035c0 2023-12-11T19:25:05.639 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.637+0000 7f08af7fe700 1 -- 172.21.15.167:0/1565671901 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f08b4034350 con 0x7f08a40035c0 2023-12-11T19:25:05.639 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:25:05.639 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","modified":"2023-12-11T19:24:59.891971Z","created":"2023-12-11T19:23:04.843190Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-11T19:25:05.641 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f089c038700 msgr2=0x7f089c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f089c038700 0x7f089c03abc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009850 tx=0x7f08a8008040 comp rx=0 tx=0).stop 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 msgr2=0x7f08b819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08b819e850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08b40059d0 tx=0x7f08b4005510 comp rx=0 tx=0).stop 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 shutdown_connections 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f089c038700 0x7f089c03abc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a40035c0 0x7f08b819e850 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:05.642 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 --2- 172.21.15.167:0/1565671901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08b819ed90 0x7f08b8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:05.643 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 >> 172.21.15.167:0/1565671901 conn(0x7f08b80fcb20 msgr2=0x7f08b80fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:05.643 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 shutdown_connections 2023-12-11T19:25:05.643 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:05.639+0000 7f08c03fc700 1 -- 172.21.15.167:0/1565671901 wait complete. 2023-12-11T19:25:05.643 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: Deploying daemon mon.b on smithi070 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/1565671901' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T19:25:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: Deploying daemon mon.b on smithi070 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/1565671901' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T19:25:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:07.184 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-11T19:25:07.185 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-11T19:25:07.964 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:25:08.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:07 smithi167 ceph-mon[102750]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:08.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:07 smithi066 ceph-mon[97920]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:08.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:08 smithi070 systemd[1]: Starting Ceph mon.b for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:25:09.411 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 podman[104004]: 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 bash[104004]: 84d5cd727a8dd8019162409f414f297a9f70e4dbcbcae12a524fdd981ac439f8 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: set uid:gid to 167:167 (ceph:ceph) 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pidfile_write: ignore empty --pid-file 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: load: jerasure load: lrc 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: RocksDB version: 7.9.2 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Git sha 0 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: DB SUMMARY 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: DB Session ID: 57OIN2IFMTT7V977BK8O 2023-12-11T19:25:09.412 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: CURRENT file: CURRENT 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: IDENTITY file: IDENTITY 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.error_if_exists: 0 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.create_if_missing: 0 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.paranoid_checks: 1 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.env: 0x557c6d693a60 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.fs: PosixFileSystem 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.info_log: 0x557c6df57780 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_file_opening_threads: 16 2023-12-11T19:25:09.413 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.statistics: (nil) 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.use_fsync: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_log_file_size: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.keep_log_file_num: 1000 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.recycle_log_file_num: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.allow_fallocate: 1 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.allow_mmap_reads: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.allow_mmap_writes: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.use_direct_reads: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.create_missing_column_families: 0 2023-12-11T19:25:09.414 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.db_log_dir: 2023-12-11T19:25:09.415 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.wal_dir: 2023-12-11T19:25:09.416 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-11T19:25:09.416 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-11T19:25:09.416 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-11T19:25:09.416 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-11T19:25:09.416 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.advise_random_on_open: 1 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.db_write_buffer_size: 0 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.write_buffer_manager: 0x557c6df0d360 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.rate_limiter: (nil) 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.wal_recovery_mode: 2 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.enable_thread_tracking: 0 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.enable_pipelined_write: 0 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.unordered_write: 0 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-11T19:25:09.417 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.row_cache: None 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.wal_filter: None 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.allow_ingest_behind: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.two_write_queues: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.manual_wal_flush: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.wal_compression: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.atomic_flush: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-11T19:25:09.418 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-11T19:25:09.419 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-11T19:25:09.419 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.log_readahead_size: 0 2023-12-11T19:25:09.419 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-11T19:25:09.419 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.best_efforts_recovery: 0 2023-12-11T19:25:09.419 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-11T19:25:09.419 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-11T19:25:09.420 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.allow_data_in_errors: 0 2023-12-11T19:25:09.420 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.db_host_id: __hostname__ 2023-12-11T19:25:09.420 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-11T19:25:09.420 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_background_jobs: 2 2023-12-11T19:25:09.420 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_background_compactions: -1 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_subcompactions: 1 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_total_wal_size: 0 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_open_files: -1 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bytes_per_sync: 0 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-11T19:25:09.421 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_readahead_size: 0 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_background_flushes: -1 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Compression algorithms supported: 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kZSTD supported: 0 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kXpressCompression supported: 0 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kLZ4HCCompression supported: 1 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kZlibCompression supported: 1 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kSnappyCompression supported: 1 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kLZ4Compression supported: 1 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: kBZip2Compression supported: 0 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-11T19:25:09.422 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.merge_operator: 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_filter: None 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_filter_factory: None 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.sst_partitioner_factory: None 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557c6df578a0) 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout: cache_index_and_filter_blocks: 1 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout: pin_top_level_index_and_filter: 1 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout: index_type: 0 2023-12-11T19:25:09.423 INFO:journalctl@ceph.mon.b.smithi070.stdout: data_block_index_type: 0 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: index_shortening: 1 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: checksum: 4 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: no_block_cache: 0 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_cache: 0x557c6dfad090 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_cache_name: BinnedLRUCache 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_cache_options: 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: capacity : 536870912 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: num_shard_bits : 4 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: strict_capacity_limit : 0 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: high_pri_pool_ratio: 0.000 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_cache_compressed: (nil) 2023-12-11T19:25:09.425 INFO:journalctl@ceph.mon.b.smithi070.stdout: persistent_cache: (nil) 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_size: 4096 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_size_deviation: 10 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_restart_interval: 16 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: index_block_restart_interval: 1 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: metadata_block_size: 4096 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: partition_filters: 0 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: use_delta_encoding: 1 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: filter_policy: bloomfilter 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: whole_key_filtering: 1 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: verify_compression: 0 2023-12-11T19:25:09.426 INFO:journalctl@ceph.mon.b.smithi070.stdout: read_amp_bytes_per_bit: 0 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: format_version: 5 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: enable_index_compression: 1 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: block_align: 0 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: max_auto_readahead_size: 262144 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: prepopulate_block_cache: 0 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: initial_auto_readahead_size: 8192 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.write_buffer_size: 33554432 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_write_buffer_number: 2 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression: NoCompression 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression: Disabled 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.prefix_extractor: nullptr 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.num_levels: 7 2023-12-11T19:25:09.427 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-11T19:25:09.428 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.level: 32767 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.strategy: 0 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.enabled: false 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-11T19:25:09.429 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.target_file_size_base: 67108864 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-11T19:25:09.430 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.arena_block_size: 1048576 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.disable_auto_compactions: 0 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-11T19:25:09.431 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.table_properties_collectors: 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.inplace_update_support: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.bloom_locality: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.max_successive_merges: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.paranoid_file_checks: 0 2023-12-11T19:25:09.432 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.force_consistency_checks: 1 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.report_bg_io_stats: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.ttl: 2592000 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.enable_blob_files: false 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.min_blob_size: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.blob_file_size: 268435456 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.blob_file_starting_level: 0 2023-12-11T19:25:09.433 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ad4de4f3-0928-460c-9ca1-4672233565bc 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322709398724, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322709399459, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702322709, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ad4de4f3-0928-460c-9ca1-4672233565bc", "db_session_id": "57OIN2IFMTT7V977BK8O", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702322709399683, "job": 1, "event": "recovery_finished"} 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-11T19:25:09.434 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557c6e048000 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: DB pointer 0x557c6e034000 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: ** DB Stats ** 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-11T19:25:09.435 INFO:journalctl@ceph.mon.b.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: ** Compaction Stats [default] ** 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: ** Compaction Stats [default] ** 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T19:25:09.436 INFO:journalctl@ceph.mon.b.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Cumulative compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Interval compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-11T19:25:09.437 INFO:journalctl@ceph.mon.b.smithi070.stdout: Block cache BinnedLRUCache@0x557c6dfad090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout: 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: using public_addr v2:172.21.15.70:0/0 -> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: starting mon.b rank -1 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(???) e0 preinit fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:25:09.438 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 systemd[1]: Started Ceph mon.b for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:25:09.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).mds e1 new map 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout: e1 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout: legacy client fscid: -1 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout: 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout: No filesystems configured 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-11T19:25:09.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Deploying cephadm binary to smithi167 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi070:/etc/ceph/ceph.conf 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Added host smithi167 2023-12-11T19:25:09.864 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3245414333' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3245414333' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:09.865 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi167:/etc/ceph/ceph.conf 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.866 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c;count:3 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Deploying daemon mon.c on smithi167 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/3690909876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.a calling monitor election 2023-12-11T19:25:09.867 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.c calling monitor election 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: fsmap 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mgrmap e13: x(active, since 54s) 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: overall HEALTH_OK 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.868 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: Deploying daemon mon.b on smithi070 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/1565671901' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:09.869 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:09 smithi070 ceph-mon[104044]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-11T19:25:14.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:25:14.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: mon.a calling monitor election 2023-12-11T19:25:14.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: mon.c calling monitor election 2023-12-11T19:25:14.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:14.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: fsmap 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: mgrmap e13: x(active, since 64s) 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: mon.b (rank 2) addr [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] is down (out of quorum) 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:25:14.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: mon.a calling monitor election 2023-12-11T19:25:14.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: mon.c calling monitor election 2023-12-11T19:25:14.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:14.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: fsmap 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: mgrmap e13: x(active, since 64s) 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-11T19:25:14.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-11T19:25:14.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: mon.b (rank 2) addr [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] is down (out of quorum) 2023-12-11T19:25:14.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:14.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 -- 172.21.15.167:0/3041479342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb8c005c30 msgr2=0x7feb8c0035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 --2- 172.21.15.167:0/3041479342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb8c005c30 0x7feb8c0035c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feba01024a0 tx=0x7feb9c00ff20 comp rx=0 tx=0).stop 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 -- 172.21.15.167:0/3041479342 shutdown_connections 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 --2- 172.21.15.167:0/3041479342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb8c005c30 0x7feb8c0035c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 -- 172.21.15.167:0/3041479342 >> 172.21.15.167:0/3041479342 conn(0x7feba00fcb20 msgr2=0x7feba00fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 -- 172.21.15.167:0/3041479342 shutdown_connections 2023-12-11T19:25:15.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.494+0000 7feba8071700 1 -- 172.21.15.167:0/3041479342 wait complete. 2023-12-11T19:25:15.497 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.495+0000 7feba8071700 1 Processor -- start 2023-12-11T19:25:15.497 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.495+0000 7feba8071700 1 -- start start 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba8071700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 0x7feba00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba8071700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 0x7feba0074290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba8071700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feba01a2b90 0x7feba01a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba8071700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01a54c0 con 0x7feba01a2b90 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba8071700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01a5630 con 0x7feba0073e10 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba8071700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01a57f0 con 0x7feb8c005c30 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba5e0d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 0x7feba00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba5e0d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 0x7feba00738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37080/0 (socket says 172.21.15.167:37080) 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba5e0d700 1 -- 172.21.15.167:0/3399408945 learned_addr learned my addr 172.21.15.167:0/3399408945 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba660e700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feba01a2b90 0x7feba01a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.496+0000 7feba560c700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 0x7feba0074290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba560c700 1 -- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 msgr2=0x7feba0074290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba560c700 1 -- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 msgr2=0x7feba0074290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba560c700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 0x7feba0074290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-11T19:25:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba560c700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 0x7feba0074290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba5e0d700 1 -- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 msgr2=0x7feba0074290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba5e0d700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 0x7feba0074290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba5e0d700 1 -- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feba01a2b90 msgr2=0x7feba01a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba5e0d700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feba01a2b90 0x7feba01a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba5e0d700 1 -- 172.21.15.167:0/3399408945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c00f970 con 0x7feb8c005c30 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba660e700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feba01a2b90 0x7feba01a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:25:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba5e0d700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 0x7feba00738d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb9c01da30 tx=0x7feb9c01da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0054e0 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb9c005640 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba01a5a70 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.497+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c02f940 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.498+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba01a5f10 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.498+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7feb9c003680 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.499+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba0102750 con 0x7feb8c005c30 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.499+0000 7feb96ffd700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feb84038d90 0x7feb8403b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:15.503 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.499+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feb9c02a070 con 0x7feb8c005c30 2023-12-11T19:25:15.504 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.499+0000 7feba560c700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feb84038d90 0x7feb8403b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:15.504 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.500+0000 7feba560c700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feb84038d90 0x7feb8403b250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb90009850 tx=0x7feb90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:15.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.505+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7feb9c02db70 con 0x7feb8c005c30 2023-12-11T19:25:15.818 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.816+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feba0108c70 con 0x7feb8c005c30 2023-12-11T19:25:15.818 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.816+0000 7feb96ffd700 1 -- 172.21.15.167:0/3399408945 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7feb9c02db70 con 0x7feb8c005c30 2023-12-11T19:25:15.818 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-12-11T19:25:15.819 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","modified":"2023-12-11T19:25:09.439128Z","created":"2023-12-11T19:23:04.843190Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-11T19:25:15.820 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feb84038d90 msgr2=0x7feb8403b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:15.820 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feb84038d90 0x7feb8403b250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feb90009850 tx=0x7feb90008040 comp rx=0 tx=0).stop 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 msgr2=0x7feba00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 0x7feba00738d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb9c01da30 tx=0x7feb9c01da60 comp rx=0 tx=0).stop 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 shutdown_connections 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feb84038d90 0x7feb8403b250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feba01a2b90 0x7feba01a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb8c005c30 0x7feba00738d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 --2- 172.21.15.167:0/3399408945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba0073e10 0x7feba0074290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 >> 172.21.15.167:0/3399408945 conn(0x7feba00fcb20 msgr2=0x7feba0104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 shutdown_connections 2023-12-11T19:25:15.821 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:15.819+0000 7feba8071700 1 -- 172.21.15.167:0/3399408945 wait complete. 2023-12-11T19:25:15.822 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2023-12-11T19:25:15.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:15 smithi167 ceph-mon[102750]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:15.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:15.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:15.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:15 smithi066 ceph-mon[97920]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:16.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:16.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:16.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.376 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-11T19:25:16.376 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph config generate-minimal-conf 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.a calling monitor election 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.c calling monitor election 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.741 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: fsmap 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mgrmap e13: x(active, since 64s) 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.b (rank 2) addr [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] is down (out of quorum) 2023-12-11T19:25:16.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.b calling monitor election 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.b calling monitor election 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.c calling monitor election 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.a calling monitor election 2023-12-11T19:25:16.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: mon.b calling monitor election 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: mon.b calling monitor election 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: mon.c calling monitor election 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: mon.a calling monitor election 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: fsmap 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: mgrmap e13: x(active, since 66s) 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: Cluster is now healthy 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: overall HEALTH_OK 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:17.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:17.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: mon.b calling monitor election 2023-12-11T19:25:17.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: mon.b calling monitor election 2023-12-11T19:25:17.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: mon.c calling monitor election 2023-12-11T19:25:17.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: mon.a calling monitor election 2023-12-11T19:25:17.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T19:25:17.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: fsmap 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: mgrmap e13: x(active, since 66s) 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: Cluster is now healthy 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: overall HEALTH_OK 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:17.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: fsmap 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: mgrmap e13: x(active, since 66s) 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: Cluster is now healthy 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: overall HEALTH_OK 2023-12-11T19:25:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:17.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:17.183 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: Updating smithi066:/etc/ceph/ceph.conf 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: Updating smithi070:/etc/ceph/ceph.conf 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: Updating smithi167:/etc/ceph/ceph.conf 2023-12-11T19:25:18.091 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:18.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:18.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: Updating smithi066:/etc/ceph/ceph.conf 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: Updating smithi070:/etc/ceph/ceph.conf 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: Updating smithi167:/etc/ceph/ceph.conf 2023-12-11T19:25:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:18.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: Updating smithi066:/etc/ceph/ceph.conf 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: Updating smithi070:/etc/ceph/ceph.conf 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: Updating smithi167:/etc/ceph/ceph.conf 2023-12-11T19:25:18.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.713+0000 7fa667e1c700 1 -- 172.21.15.66:0/1624021249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa660106f80 msgr2=0x7fa660107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.713+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1624021249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa660106f80 0x7fa660107360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009910 tx=0x7fa65c009c20 comp rx=0 tx=0).stop 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.713+0000 7fa667e1c700 1 -- 172.21.15.66:0/1624021249 shutdown_connections 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.714+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1624021249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa660106f80 0x7fa660107360 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.714+0000 7fa667e1c700 1 -- 172.21.15.66:0/1624021249 >> 172.21.15.66:0/1624021249 conn(0x7fa660075930 msgr2=0x7fa660075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.714+0000 7fa667e1c700 1 -- 172.21.15.66:0/1624021249 shutdown_connections 2023-12-11T19:25:18.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.714+0000 7fa667e1c700 1 -- 172.21.15.66:0/1624021249 wait complete. 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 Processor -- start 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 -- start start 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa660106f80 0x7fa66019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 0x7fa660198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa660198e80 0x7fa660199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66019f340 con 0x7fa66019ed40 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6601a5620 con 0x7fa660198e80 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa667e1c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6601a5760 con 0x7fa660106f80 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.715+0000 7fa6653b7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 0x7fa660198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 0x7fa660198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36818/0 (socket says 172.21.15.66:36818) 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 -- 172.21.15.66:0/1172895575 learned_addr learned my addr 172.21.15.66:0/1172895575 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:25:18.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa665bb8700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa660106f80 0x7fa66019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6663b9700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa660198e80 0x7fa660199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 -- 172.21.15.66:0/1172895575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa660106f80 msgr2=0x7fa66019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa660106f80 0x7fa66019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 -- 172.21.15.66:0/1172895575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa660198e80 msgr2=0x7fa660199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa660198e80 0x7fa660199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 -- 172.21.15.66:0/1172895575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0095f0 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.716+0000 7fa6653b7700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 0x7fa660198940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b7b0 tx=0x7fa65000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.717+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650004230 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.717+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6601a5a40 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.717+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa650004390 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.717+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650011a50 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.717+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6601a5f40 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.718+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fa650011bb0 con 0x7fa66019ed40 2023-12-11T19:25:18.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.718+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6600ffd70 con 0x7fa66019ed40 2023-12-11T19:25:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.718+0000 7fa656ffd700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa64c038d90 0x7fa64c03b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.719+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa65004aed0 con 0x7fa66019ed40 2023-12-11T19:25:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.719+0000 7fa665bb8700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa64c038d90 0x7fa64c03b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:18.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.719+0000 7fa665bb8700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa64c038d90 0x7fa64c03b250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa65c004690 tx=0x7fa65c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:18.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.724+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa6500036a0 con 0x7fa66019ed40 2023-12-11T19:25:18.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.946+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa660199e00 con 0x7fa66019ed40 2023-12-11T19:25:18.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.947+0000 7fa656ffd700 1 -- 172.21.15.66:0/1172895575 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7fa650011e60 con 0x7fa66019ed40 2023-12-11T19:25:18.948 INFO:teuthology.orchestra.run.smithi066.stdout:# minimal ceph.conf for 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:25:18.948 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2023-12-11T19:25:18.948 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:25:18.948 INFO:teuthology.orchestra.run.smithi066.stdout: mon_host = [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2023-12-11T19:25:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa64c038d90 msgr2=0x7fa64c03b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:18.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa64c038d90 0x7fa64c03b250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa65c004690 tx=0x7fa65c005fb0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 msgr2=0x7fa660198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 0x7fa660198940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b7b0 tx=0x7fa65000bac0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 shutdown_connections 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa64c038d90 0x7fa64c03b250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa660198e80 0x7fa660199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa660106f80 0x7fa66019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 --2- 172.21.15.66:0/1172895575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa66019ed40 0x7fa660198940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.949+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 >> 172.21.15.66:0/1172895575 conn(0x7fa660075930 msgr2=0x7fa660103410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.950+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 shutdown_connections 2023-12-11T19:25:18.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:18.950+0000 7fa667e1c700 1 -- 172.21.15.66:0/1172895575 wait complete. 2023-12-11T19:25:19.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: Updating smithi066:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.180 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:19.181 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:19.181 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:19.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: Updating smithi066:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:19.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: Updating smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: Updating smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: Updating smithi066:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T19:25:19.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:19.590 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-11T19:25:19.590 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:25:19.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2023-12-11T19:25:19.664 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:25:19.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:19.776 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:25:19.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2023-12-11T19:25:19.853 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:25:19.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:19.964 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:25:19.964 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2023-12-11T19:25:20.043 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:25:20.043 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T19:25:20.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:19 smithi066 ceph-mon[97920]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:20.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:19 smithi066 ceph-mon[97920]: Reconfiguring mon.a (unknown last config time)... 2023-12-11T19:25:20.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:19 smithi066 ceph-mon[97920]: Reconfiguring daemon mon.a on smithi066 2023-12-11T19:25:20.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:19 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1172895575' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:20.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:19 smithi070 ceph-mon[104044]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:20.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:19 smithi070 ceph-mon[104044]: Reconfiguring mon.a (unknown last config time)... 2023-12-11T19:25:20.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:19 smithi070 ceph-mon[104044]: Reconfiguring daemon mon.a on smithi066 2023-12-11T19:25:20.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:19 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1172895575' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:20.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:19 smithi167 ceph-mon[102750]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:20.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:19 smithi167 ceph-mon[102750]: Reconfiguring mon.a (unknown last config time)... 2023-12-11T19:25:20.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:19 smithi167 ceph-mon[102750]: Reconfiguring daemon mon.a on smithi066 2023-12-11T19:25:20.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:19 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1172895575' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:20.153 INFO:tasks.cephadm:Adding mgr.x on smithi066 2023-12-11T19:25:20.153 INFO:tasks.cephadm:Adding mgr.y on smithi070 2023-12-11T19:25:20.153 INFO:tasks.cephadm:Adding mgr.z on smithi167 2023-12-11T19:25:20.153 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch apply mgr '3;smithi066=x;smithi070=y;smithi167=z' 2023-12-11T19:25:20.961 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:25:21.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:21.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:21.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: Reconfiguring mon.b (monmap changed)... 2023-12-11T19:25:21.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:21.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:21.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:21.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:20 smithi070 ceph-mon[104044]: Reconfiguring daemon mon.b on smithi070 2023-12-11T19:25:21.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:21.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:21.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: Reconfiguring mon.b (monmap changed)... 2023-12-11T19:25:21.258 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:21.258 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:21.258 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:21.258 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:20 smithi167 ceph-mon[102750]: Reconfiguring daemon mon.b on smithi070 2023-12-11T19:25:21.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:21.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:21.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: Reconfiguring mon.b (monmap changed)... 2023-12-11T19:25:21.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:21.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:21.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:21.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:20 smithi066 ceph-mon[97920]: Reconfiguring daemon mon.b on smithi070 2023-12-11T19:25:22.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:22 smithi167 ceph-mon[102750]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:22.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:22.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:22.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:22.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:22.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:22.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:22 smithi066 ceph-mon[97920]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:22.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:22.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:22.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:22.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:22.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:22.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:22 smithi070 ceph-mon[104044]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:22.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:22.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:22.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T19:25:22.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T19:25:22.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 -- 172.21.15.167:0/3135527576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 msgr2=0x7f9b60101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3135527576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 0x7f9b60101480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009910 tx=0x7f9b50009c20 comp rx=0 tx=0).stop 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 -- 172.21.15.167:0/3135527576 shutdown_connections 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3135527576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 0x7f9b60101480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 -- 172.21.15.167:0/3135527576 >> 172.21.15.167:0/3135527576 conn(0x7f9b600fc620 msgr2=0x7f9b600fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 -- 172.21.15.167:0/3135527576 shutdown_connections 2023-12-11T19:25:22.396 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.395+0000 7f9b6617d700 1 -- 172.21.15.167:0/3135527576 wait complete. 2023-12-11T19:25:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 Processor -- start 2023-12-11T19:25:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 -- start start 2023-12-11T19:25:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 0x7f9b60196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b60196a60 0x7f9b6019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 0x7f9b6019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:22.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6019e7b0 con 0x7f9b60101060 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6019e920 con 0x7f9b60196a60 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.396+0000 7f9b6617d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6019eae0 con 0x7f9b60196ee0 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b60196a60 0x7f9b6019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 0x7f9b6019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b60196a60 0x7f9b6019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.167:45050/0 (socket says 172.21.15.167:45050) 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5effd700 1 -- 172.21.15.167:0/3656319393 learned_addr learned my addr 172.21.15.167:0/3656319393 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 0x7f9b6019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60132/0 (socket says 172.21.15.167:60132) 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5f7fe700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 0x7f9b60196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 -- 172.21.15.167:0/3656319393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b60196a60 msgr2=0x7f9b6019ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:22.400 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b60196a60 0x7f9b6019ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 -- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 msgr2=0x7f9b60196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 0x7f9b60196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 -- 172.21.15.167:0/3656319393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b500095f0 con 0x7f9b60196ee0 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.397+0000 7f9b5ffff700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 0x7f9b6019e010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800bd50 tx=0x7f9b5800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.398+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800cbb0 con 0x7f9b60196ee0 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.398+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b5800cd10 con 0x7f9b60196ee0 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.398+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6019edc0 con 0x7f9b60196ee0 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.398+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580125c0 con 0x7f9b60196ee0 2023-12-11T19:25:22.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.398+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6019f2e0 con 0x7f9b60196ee0 2023-12-11T19:25:22.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.399+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b60062c00 con 0x7f9b60196ee0 2023-12-11T19:25:22.403 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.399+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f9b58014440 con 0x7f9b60196ee0 2023-12-11T19:25:22.403 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.400+0000 7f9b5cff9700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b48038d90 0x7f9b4803b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:22.403 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.400+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9b5804ac80 con 0x7f9b60196ee0 2023-12-11T19:25:22.403 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.400+0000 7f9b5f7fe700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b48038d90 0x7f9b4803b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:22.403 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.401+0000 7f9b5f7fe700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b48038d90 0x7f9b4803b250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009fd0 tx=0x7f9b50005160 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:22.406 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.405+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b58051080 con 0x7f9b60196ee0 2023-12-11T19:25:22.651 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.650+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi070=y;smithi167=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b600fe630 con 0x7f9b48038d90 2023-12-11T19:25:22.657 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.657+0000 7f9b5cff9700 1 -- 172.21.15.167:0/3656319393 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9b600fe630 con 0x7f9b48038d90 2023-12-11T19:25:22.658 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b48038d90 msgr2=0x7f9b4803b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b48038d90 0x7f9b4803b250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009fd0 tx=0x7f9b50005160 comp rx=0 tx=0).stop 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 msgr2=0x7f9b6019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 0x7f9b6019e010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800bd50 tx=0x7f9b5800bd80 comp rx=0 tx=0).stop 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 shutdown_connections 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b48038d90 0x7f9b4803b250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b60196ee0 0x7f9b6019e010 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b60101060 0x7f9b60196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.660 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 --2- 172.21.15.167:0/3656319393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b60196a60 0x7f9b6019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:22.661 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 >> 172.21.15.167:0/3656319393 conn(0x7f9b600fc620 msgr2=0x7f9b600fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:22.661 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 shutdown_connections 2023-12-11T19:25:22.661 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:25:22.659+0000 7f9b6617d700 1 -- 172.21.15.167:0/3656319393 wait complete. 2023-12-11T19:25:23.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:23 smithi167 ceph-mon[102750]: Reconfiguring mon.c (monmap changed)... 2023-12-11T19:25:23.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:23 smithi167 ceph-mon[102750]: Reconfiguring daemon mon.c on smithi167 2023-12-11T19:25:23.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:23 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:23.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:23 smithi066 ceph-mon[97920]: Reconfiguring mon.c (monmap changed)... 2023-12-11T19:25:23.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:23 smithi066 ceph-mon[97920]: Reconfiguring daemon mon.c on smithi167 2023-12-11T19:25:23.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:23.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:23 smithi070 ceph-mon[104044]: Reconfiguring mon.c (monmap changed)... 2023-12-11T19:25:23.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:23 smithi070 ceph-mon[104044]: Reconfiguring daemon mon.c on smithi167 2023-12-11T19:25:23.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:23 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:23.655 DEBUG:teuthology.orchestra.run.smithi070:mgr.y> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.y.service 2023-12-11T19:25:23.659 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.z.service 2023-12-11T19:25:23.665 INFO:tasks.cephadm:Deploying OSDs... 2023-12-11T19:25:23.665 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:25:23.665 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2023-12-11T19:25:23.717 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-11T19:25:23.717 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2023-12-11T19:25:23.735 INFO:journalctl@ceph.mgr.y.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:25:23.740 INFO:journalctl@ceph.mgr.z.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:25:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-11T19:25:23.767 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:23.767 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-12-11T19:25:23.767 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:23.767 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:23.767 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-12-11 19:24:20.320651089 +0000 2023-12-11T19:25:23.768 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-12-11 19:18:26.302500709 +0000 2023-12-11T19:25:23.768 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-12-11 19:18:26.302500709 +0000 2023-12-11T19:25:23.768 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-12-11 19:18:26.271500931 +0000 2023-12-11T19:25:23.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-11T19:25:23.844 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-12-11T19:25:23.844 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-12-11T19:25:23.844 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000310592 s, 1.6 MB/s 2023-12-11T19:25:23.847 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-11T19:25:23.893 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2023-12-11T19:25:23.978 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-12-11 19:24:20.383650623 +0000 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-12-11 19:18:27.146494647 +0000 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-12-11 19:18:27.146494647 +0000 2023-12-11T19:25:23.979 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-12-11 19:18:27.118494848 +0000 2023-12-11T19:25:23.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-11T19:25:24.087 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-12-11T19:25:24.087 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-12-11T19:25:24.087 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000292249 s, 1.8 MB/s 2023-12-11T19:25:24.087 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:24.087 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi070=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:24.087 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: Saving service mgr spec with placement smithi066=x;smithi070=y;smithi167=z;count:3 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:24.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:24.089 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-11T19:25:24.136 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2023-12-11T19:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-11T19:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-12-11T19:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-12-11 19:24:20.461650046 +0000 2023-12-11T19:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-12-11 19:18:27.990488586 +0000 2023-12-11T19:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-12-11 19:18:27.990488586 +0000 2023-12-11T19:25:24.222 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-12-11 19:18:27.963488780 +0000 2023-12-11T19:25:24.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-11T19:25:24.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi070=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: Saving service mgr spec with placement smithi066=x;smithi070=y;smithi167=z;count:3 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:24.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:24.329 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-12-11T19:25:24.329 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-12-11T19:25:24.329 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000303277 s, 1.7 MB/s 2023-12-11T19:25:24.332 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-11T19:25:24.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:24.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi070=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: Saving service mgr spec with placement smithi066=x;smithi070=y;smithi167=z;count:3 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:24.378 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2023-12-11T19:25:24.423 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-11T19:25:24.423 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-12-11 19:24:20.528649551 +0000 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-12-11 19:18:28.831482546 +0000 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-12-11 19:18:28.831482546 +0000 2023-12-11T19:25:24.424 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-12-11 19:18:28.801482761 +0000 2023-12-11T19:25:24.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-11T19:25:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-12-11T19:25:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-12-11T19:25:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000299177 s, 1.7 MB/s 2023-12-11T19:25:24.534 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-11T19:25:24.580 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-12-11T19:25:24.581 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2023-12-11T19:25:24.628 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-11T19:25:24.629 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-12-11 19:25:08.505933280 +0000 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-12-11 19:17:56.795850380 +0000 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-12-11 19:17:56.795850380 +0000 2023-12-11T19:25:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-12-11 19:17:56.772851047 +0000 2023-12-11T19:25:24.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-11T19:25:24.823 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-12-11T19:25:24.823 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-12-11T19:25:24.823 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000201562 s, 2.5 MB/s 2023-12-11T19:25:24.825 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-11T19:25:24.871 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-12-11 19:25:08.505933280 +0000 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-12-11 19:17:57.609826777 +0000 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-12-11 19:17:57.609826777 +0000 2023-12-11T19:25:24.955 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-12-11 19:17:57.586827444 +0000 2023-12-11T19:25:24.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-11T19:25:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-12-11T19:25:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-12-11T19:25:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000171539 s, 3.0 MB/s 2023-12-11T19:25:25.062 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-11T19:25:25.107 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2023-12-11T19:25:25.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:25 smithi070 ceph-mon[104044]: Deploying daemon mgr.z on smithi167 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-12-11 19:25:08.505933280 +0000 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-12-11 19:17:58.449802421 +0000 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-12-11 19:17:58.449802421 +0000 2023-12-11T19:25:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-12-11 19:17:58.427803059 +0000 2023-12-11T19:25:25.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-11T19:25:25.257 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-12-11T19:25:25.258 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-12-11T19:25:25.258 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.00020776 s, 2.5 MB/s 2023-12-11T19:25:25.260 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-11T19:25:25.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:25 smithi167 ceph-mon[102750]: Deploying daemon mgr.z on smithi167 2023-12-11T19:25:25.304 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2023-12-11T19:25:25.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:25 smithi066 ceph-mon[97920]: Deploying daemon mgr.z on smithi167 2023-12-11T19:25:25.387 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-12-11 19:25:08.505933280 +0000 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-12-11 19:17:59.258778964 +0000 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-12-11 19:17:59.258778964 +0000 2023-12-11T19:25:25.388 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-12-11 19:17:59.236779602 +0000 2023-12-11T19:25:25.388 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-11T19:25:25.492 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-12-11T19:25:25.492 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-12-11T19:25:25.493 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000183542 s, 2.8 MB/s 2023-12-11T19:25:25.495 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-11T19:25:25.540 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-12-11T19:25:25.540 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2023-12-11T19:25:25.591 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-11T19:25:25.591 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-12-11 19:25:25.481185699 +0000 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-12-11 19:17:56.824737280 +0000 2023-12-11T19:25:25.676 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-12-11 19:17:56.824737280 +0000 2023-12-11T19:25:25.677 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2023-12-11 19:17:56.795737777 +0000 2023-12-11T19:25:25.677 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-11T19:25:25.938 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-12-11T19:25:25.938 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-12-11T19:25:25.938 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000317698 s, 1.6 MB/s 2023-12-11T19:25:25.940 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-11T19:25:25.987 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2023-12-11T19:25:26.036 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-12-11 19:25:25.805180119 +0000 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-12-11 19:17:57.650723138 +0000 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-12-11 19:17:57.650723138 +0000 2023-12-11T19:25:26.037 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2023-12-11 19:17:57.621723635 +0000 2023-12-11T19:25:26.037 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-11T19:25:26.107 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-12-11T19:25:26.108 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-12-11T19:25:26.108 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000309392 s, 1.7 MB/s 2023-12-11T19:25:26.110 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-11T19:25:26.157 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2023-12-11T19:25:26.243 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-12-11 19:25:25.805180119 +0000 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-12-11 19:17:58.463709219 +0000 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-12-11 19:17:58.463709219 +0000 2023-12-11T19:25:26.244 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2023-12-11 19:17:58.436709681 +0000 2023-12-11T19:25:26.244 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-11T19:25:26.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:26 smithi167 ceph-mon[102750]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:26.271 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:25 smithi167 systemd[1]: Starting Ceph mgr.z for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:25:26.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:26 smithi066 ceph-mon[97920]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:26.342 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-12-11T19:25:26.342 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-12-11T19:25:26.342 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000290935 s, 1.8 MB/s 2023-12-11T19:25:26.345 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-11T19:25:26.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:26 smithi070 ceph-mon[104044]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:26.390 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2023-12-11T19:25:26.475 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-11T19:25:26.475 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T19:25:26.475 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-12-11T19:25:26.475 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:25:26.475 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T19:25:26.476 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-12-11 19:25:25.805180119 +0000 2023-12-11T19:25:26.476 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-12-11 19:17:59.281695214 +0000 2023-12-11T19:25:26.476 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-12-11 19:17:59.281695214 +0000 2023-12-11T19:25:26.476 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2023-12-11 19:17:59.254695676 +0000 2023-12-11T19:25:26.476 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-11T19:25:26.586 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-12-11T19:25:26.586 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-12-11T19:25:26.586 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000307418 s, 1.7 MB/s 2023-12-11T19:25:26.586 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:26 smithi167 podman[104746]: 2023-12-11T19:25:26.588 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-11T19:25:26.634 INFO:tasks.cephadm:Deploying osd.0 on smithi066 with /dev/vg_nvme/lv_4... 2023-12-11T19:25:26.634 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-11T19:25:26.861 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:26 smithi167 bash[104746]: 8f55d6791a3d437ec1e46772de76c8d48575997bb891b103315c4de1c32a6043 2023-12-11T19:25:26.861 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:26 smithi167 ceph-mgr[104891]: -- 172.21.15.167:0/764427943 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557037d9d520 con 0x557037d74800 2023-12-11T19:25:26.861 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:26 smithi167 systemd[1]: Started Ceph mgr.z for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:25:26.966 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:26.860+0000 7fc4c4708200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T19:25:27.201 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:26.965+0000 7fc4c4708200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T19:25:27.201 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:27.080+0000 7fc4c4708200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T19:25:27.201 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:27.200+0000 7fc4c4708200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T19:25:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:25:27.770 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:27.418+0000 7fc4c4708200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:27.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:27 smithi066 ceph-mon[97920]: Deploying daemon mgr.y on smithi070 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:28.135 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:27 smithi070 ceph-mon[104044]: Deploying daemon mgr.y on smithi070 2023-12-11T19:25:28.162 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:27.879+0000 7fc4c4708200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T19:25:28.162 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:28.047+0000 7fc4c4708200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:28.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:27 smithi167 ceph-mon[102750]: Deploying daemon mgr.y on smithi070 2023-12-11T19:25:28.521 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:28.161+0000 7fc4c4708200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T19:25:28.521 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:28.284+0000 7fc4c4708200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T19:25:29.110 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:29 smithi070 systemd[1]: Starting Ceph mgr.y for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:25:29.270 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:28.824+0000 7fc4c4708200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T19:25:29.454 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-11T19:25:29.454 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-11T19:25:29.455 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-12-11T19:25:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-12-11T19:25:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0309423 s, 339 MB/s 2023-12-11T19:25:29.455 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-12-11T19:25:29.812 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:29 smithi070 podman[105478]: 2023-12-11T19:25:29.812 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:29 smithi070 bash[105478]: bf015c9dc388740e7df728e55996d8da5cf036bc5f110f5525093354eaa74106 2023-12-11T19:25:29.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2023-12-11T19:25:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:29 smithi167 ceph-mon[102750]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:30.021 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:29.712+0000 7fc4c4708200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T19:25:30.021 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:29.818+0000 7fc4c4708200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T19:25:30.104 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:29 smithi070 ceph-mon[104044]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:30.104 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:29 smithi070 ceph-mgr[105518]: -- 172.21.15.70:0/3521556277 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55911e7d5520 con 0x55911e7ac800 2023-12-11T19:25:30.104 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:29 smithi070 systemd[1]: Started Ceph mgr.y for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:25:30.104 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:29.998+0000 7fb6b76e3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T19:25:30.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:29 smithi066 ceph-mon[97920]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:30.361 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:30.102+0000 7fb6b76e3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T19:25:30.362 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:30.229+0000 7fb6b76e3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T19:25:30.362 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:30.350+0000 7fb6b76e3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T19:25:30.737 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:25:30.860 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:30.589+0000 7fb6b76e3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T19:25:31.182 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:30.811+0000 7fc4c4708200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:25:31.182 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:30.938+0000 7fc4c4708200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T19:25:31.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:31.278 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:31.081+0000 7fb6b76e3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T19:25:31.278 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:31.276+0000 7fb6b76e3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T19:25:31.279 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.279 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.279 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.279 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.279 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.279 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:31.290 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.291 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.291 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.291 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.291 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:31.291 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:31.520 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:31.181+0000 7fc4c4708200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:25:31.562 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:31.414+0000 7fb6b76e3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T19:25:31.563 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:31.561+0000 7fb6b76e3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T19:25:31.997 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:31.738+0000 7fc4c4708200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T19:25:31.997 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:31.862+0000 7fc4c4708200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 -- 172.21.15.66:0/3970506839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524101620 msgr2=0x7f6524101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 --2- 172.21.15.66:0/3970506839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524101620 0x7f6524101a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65140099c0 tx=0x7f6514009cd0 comp rx=0 tx=0).stop 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 -- 172.21.15.66:0/3970506839 shutdown_connections 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 --2- 172.21.15.66:0/3970506839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65241031c0 0x7f6524109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 --2- 172.21.15.66:0/3970506839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524101620 0x7f6524101a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 --2- 172.21.15.66:0/3970506839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6524102820 0x7f6524102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 -- 172.21.15.66:0/3970506839 >> 172.21.15.66:0/3970506839 conn(0x7f65240fcbe0 msgr2=0x7f65240ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 -- 172.21.15.66:0/3970506839 shutdown_connections 2023-12-11T19:25:32.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.151+0000 7f65296fb700 1 -- 172.21.15.66:0/3970506839 wait complete. 2023-12-11T19:25:32.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.152+0000 7f65296fb700 1 Processor -- start 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.152+0000 7f65296fb700 1 -- start start 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65296fb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6524101620 0x7f65241974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65296fb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 0x7f6524197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65296fb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65241031c0 0x7f652419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65296fb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652419f030 con 0x7f6524102820 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65296fb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652419f170 con 0x7f65241031c0 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65296fb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652419f300 con 0x7f6524101620 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65227fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 0x7f6524197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65227fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 0x7f6524197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56154/0 (socket says 172.21.15.66:56154) 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65227fc700 1 -- 172.21.15.66:0/2203528873 learned_addr learned my addr 172.21.15.66:0/2203528873 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f6522ffd700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6524101620 0x7f65241974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.153+0000 7f65237fe700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65241031c0 0x7f652419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65227fc700 1 -- 172.21.15.66:0/2203528873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6524101620 msgr2=0x7f65241974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65227fc700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6524101620 0x7f65241974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:32.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65227fc700 1 -- 172.21.15.66:0/2203528873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65241031c0 msgr2=0x7f652419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:32.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65227fc700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65241031c0 0x7f652419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:32.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65227fc700 1 -- 172.21.15.66:0/2203528873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65140095f0 con 0x7f6524102820 2023-12-11T19:25:32.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65227fc700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 0x7f6524197a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f651800e9d0 tx=0x7f651800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518009790 con 0x7f6524102820 2023-12-11T19:25:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f651800ee70 con 0x7f6524102820 2023-12-11T19:25:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652419f5e0 con 0x7f6524102820 2023-12-11T19:25:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518018420 con 0x7f6524102820 2023-12-11T19:25:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.154+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652419fb30 con 0x7f6524102820 2023-12-11T19:25:32.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.155+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6524108b40 con 0x7f6524102820 2023-12-11T19:25:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.155+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f6518010430 con 0x7f6524102820 2023-12-11T19:25:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.156+0000 7f650bfff700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f650c038af0 0x7f650c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:25:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.156+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6518014070 con 0x7f6524102820 2023-12-11T19:25:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.156+0000 7f6522ffd700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f650c038af0 0x7f650c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:25:32.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.157+0000 7f6522ffd700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f650c038af0 0x7f650c03afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:25:32.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.161+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f651800cbd0 con 0x7f6524102820 2023-12-11T19:25:32.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:31 smithi167 ceph-mon[102750]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:32.271 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:32.082+0000 7fc4c4708200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T19:25:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:31 smithi066 ceph-mon[97920]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:31 smithi070 ceph-mon[104044]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:32.361 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:32.142+0000 7fb6b76e3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T19:25:32.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:32.387+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6524061a10 con 0x7f650c038af0 2023-12-11T19:25:32.770 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:32.447+0000 7fc4c4708200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T19:25:33.360 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:33.078+0000 7fb6b76e3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T19:25:33.360 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:33.183+0000 7fb6b76e3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T19:25:33.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: Reconfiguring mgr.x (unknown last config time)... 2023-12-11T19:25:33.738 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:33.738 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:33.738 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.738 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:33 smithi066 ceph-mon[97920]: Reconfiguring daemon mgr.x on smithi066 2023-12-11T19:25:33.770 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:33.385+0000 7fc4c4708200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:33.557+0000 7fc4c4708200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: Reconfiguring mgr.x (unknown last config time)... 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-mon[102750]: Reconfiguring daemon mgr.x on smithi066 2023-12-11T19:25:33.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:33.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: Reconfiguring mgr.x (unknown last config time)... 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T19:25:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:33.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:33 smithi070 ceph-mon[104044]: Reconfiguring daemon mgr.x on smithi066 2023-12-11T19:25:34.227 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:33.806+0000 7fc4c4708200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T19:25:34.227 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:33.950+0000 7fc4c4708200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T19:25:34.481 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:34.138+0000 7fb6b76e3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:25:34.482 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:34.256+0000 7fb6b76e3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T19:25:34.521 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:34.226+0000 7fc4c4708200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T19:25:35.316 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:34.480+0000 7fb6b76e3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T19:25:35.316 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:35.010+0000 7fb6b76e3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T19:25:35.317 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:35.125+0000 7fb6b76e3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T19:25:35.610 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:35.316+0000 7fb6b76e3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T19:25:35.767 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z[104833]: 2023-12-11T19:25:35.303+0000 7fc4c4708200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T19:25:35.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:35.800+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 14) v1 ==== 90793+0+0 (secure 0 0 0) 0x7f6518018900 con 0x7f6524102820 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: Standby manager daemon z started 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T19:25:36.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T19:25:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:35 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: Standby manager daemon z started 2023-12-11T19:25:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T19:25:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T19:25:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T19:25:36.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:35 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T19:25:36.110 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:35.658+0000 7fb6b76e3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T19:25:36.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:36.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: Standby manager daemon z started 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T19:25:36.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:35 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.167:0/1356714816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T19:25:36.860 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:36.582+0000 7fb6b76e3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T19:25:36.860 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:36.753+0000 7fb6b76e3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T19:25:36.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-mon[104044]: mgrmap e14: x(active, since 85s), standbys: z 2023-12-11T19:25:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T19:25:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/459845591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5da1c6d-b901-47aa-b2e3-4e377f852890"}]: dispatch 2023-12-11T19:25:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/459845591' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5da1c6d-b901-47aa-b2e3-4e377f852890"}]': finished 2023-12-11T19:25:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-mon[104044]: osdmap e5: 1 total, 0 up, 1 in 2023-12-11T19:25:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:37.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:36 smithi066 ceph-mon[97920]: mgrmap e14: x(active, since 85s), standbys: z 2023-12-11T19:25:37.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T19:25:37.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:36 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/459845591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5da1c6d-b901-47aa-b2e3-4e377f852890"}]: dispatch 2023-12-11T19:25:37.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:36 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/459845591' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5da1c6d-b901-47aa-b2e3-4e377f852890"}]': finished 2023-12-11T19:25:37.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:36 smithi066 ceph-mon[97920]: osdmap e5: 1 total, 0 up, 1 in 2023-12-11T19:25:37.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:37.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:36 smithi167 ceph-mon[102750]: mgrmap e14: x(active, since 85s), standbys: z 2023-12-11T19:25:37.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T19:25:37.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:36 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/459845591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5da1c6d-b901-47aa-b2e3-4e377f852890"}]: dispatch 2023-12-11T19:25:37.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:36 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/459845591' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5da1c6d-b901-47aa-b2e3-4e377f852890"}]': finished 2023-12-11T19:25:37.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:36 smithi167 ceph-mon[102750]: osdmap e5: 1 total, 0 up, 1 in 2023-12-11T19:25:37.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:37.360 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:37.004+0000 7fb6b76e3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T19:25:37.360 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:37.149+0000 7fb6b76e3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T19:25:37.805 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:37.426+0000 7fb6b76e3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T19:25:38.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:37 smithi066 ceph-mon[97920]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:38.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:37 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3440449287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:25:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:37 smithi070 ceph-mon[104044]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:37 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3440449287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:25:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:37 smithi167 ceph-mon[102750]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:37 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3440449287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:25:38.814 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:25:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y[105494]: 2023-12-11T19:25:38.494+0000 7fb6b76e3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T19:25:38.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:38.818+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f6518017bf0 con 0x7f6524102820 2023-12-11T19:25:39.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:38 smithi066 ceph-mon[97920]: Standby manager daemon y started 2023-12-11T19:25:39.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:38 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T19:25:39.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:38 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T19:25:39.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:38 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T19:25:39.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:38 smithi066 ceph-mon[97920]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T19:25:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:38 smithi070 ceph-mon[104044]: Standby manager daemon y started 2023-12-11T19:25:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:38 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T19:25:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:38 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T19:25:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:38 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T19:25:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:38 smithi070 ceph-mon[104044]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T19:25:39.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:38 smithi167 ceph-mon[102750]: Standby manager daemon y started 2023-12-11T19:25:39.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:38 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T19:25:39.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:38 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T19:25:39.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:38 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T19:25:39.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:38 smithi167 ceph-mon[102750]: from='mgr.? 172.21.15.70:0/3864001772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T19:25:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:39 smithi070 ceph-mon[104044]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:39 smithi070 ceph-mon[104044]: mgrmap e15: x(active, since 88s), standbys: y, z 2023-12-11T19:25:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:39 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T19:25:40.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:39 smithi167 ceph-mon[102750]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:40.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:39 smithi167 ceph-mon[102750]: mgrmap e15: x(active, since 88s), standbys: y, z 2023-12-11T19:25:40.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:39 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T19:25:40.290 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:39 smithi066 ceph-mon[97920]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:40.290 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:39 smithi066 ceph-mon[97920]: mgrmap e15: x(active, since 88s), standbys: y, z 2023-12-11T19:25:40.290 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:39 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T19:25:42.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:41 smithi070 ceph-mon[104044]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:42.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:41 smithi066 ceph-mon[97920]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:42.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:41 smithi167 ceph-mon[102750]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:43 smithi070 ceph-mon[104044]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:43 smithi167 ceph-mon[102750]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:43 smithi066 ceph-mon[97920]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:45.090 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-11T19:25:45.090 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:45.090 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:44 smithi066 ceph-mon[97920]: Deploying daemon osd.0 on smithi066 2023-12-11T19:25:45.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-11T19:25:45.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:45.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:44 smithi070 ceph-mon[104044]: Deploying daemon osd.0 on smithi066 2023-12-11T19:25:45.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-11T19:25:45.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:45.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:44 smithi167 ceph-mon[102750]: Deploying daemon osd.0 on smithi066 2023-12-11T19:25:46.092 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:45 smithi066 ceph-mon[97920]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:45 smithi070 ceph-mon[104044]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:46.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:45 smithi167 ceph-mon[102750]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:48.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:47 smithi070 ceph-mon[104044]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:48.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:47 smithi167 ceph-mon[102750]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:48.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:47 smithi066 ceph-mon[97920]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:49 smithi070 ceph-mon[104044]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:50.161 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:49 smithi066 ceph-mon[97920]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:50.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:49 smithi167 ceph-mon[102750]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:51 smithi066 ceph-mon[97920]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:51 smithi070 ceph-mon[104044]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:51.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:52.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:51 smithi167 ceph-mon[102750]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:52.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:52.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:52.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:54.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:53 smithi066 ceph-mon[97920]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:54.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:54.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:54.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.199+0000 7f650bfff700 1 -- 172.21.15.66:0/2203528873 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6524061a10 con 0x7f650c038af0 2023-12-11T19:25:54.202 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 0 on host 'smithi066' 2023-12-11T19:25:54.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f650c038af0 msgr2=0x7f650c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f650c038af0 0x7f650c03afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514003680 comp rx=0 tx=0).stop 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 msgr2=0x7f6524197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 0x7f6524197a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f651800e9d0 tx=0x7f651800ece0 comp rx=0 tx=0).stop 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 shutdown_connections 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f650c038af0 0x7f650c03afb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65241031c0 0x7f652419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6524101620 0x7f65241974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 --2- 172.21.15.66:0/2203528873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6524102820 0x7f6524197a30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.201+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 >> 172.21.15.66:0/2203528873 conn(0x7f65240fcbe0 msgr2=0x7f6524105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.202+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 shutdown_connections 2023-12-11T19:25:54.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:25:54.202+0000 7f65296fb700 1 -- 172.21.15.66:0/2203528873 wait complete. 2023-12-11T19:25:54.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:53 smithi167 ceph-mon[102750]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:53 smithi070 ceph-mon[104044]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:54.681 DEBUG:teuthology.orchestra.run.smithi066:osd.0> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.0.service 2023-12-11T19:25:54.685 INFO:tasks.cephadm:Deploying osd.1 on smithi070 with /dev/vg_nvme/lv_4... 2023-12-11T19:25:54.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-11T19:25:54.762 INFO:journalctl@ceph.osd.0.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:25:55.405 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:55.405 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:55.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:55.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:55 smithi066 ceph-mon[97920]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:55.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:55 smithi066 ceph-mon[97920]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-11T19:25:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:25:55.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:55.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:55.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:55.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:55 smithi167 ceph-mon[102750]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:55.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:55 smithi167 ceph-mon[102750]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-11T19:25:55.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:55.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:55.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:25:55.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:55 smithi070 ceph-mon[104044]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:55.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:55 smithi070 ceph-mon[104044]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-11T19:25:56.296 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:56 smithi070 ceph-mon[104044]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-11T19:25:56.296 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:56 smithi070 ceph-mon[104044]: osdmap e6: 1 total, 0 up, 1 in 2023-12-11T19:25:56.296 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:56 smithi070 ceph-mon[104044]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:25:56.296 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:56 smithi066 ceph-mon[97920]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-11T19:25:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:56 smithi066 ceph-mon[97920]: osdmap e6: 1 total, 0 up, 1 in 2023-12-11T19:25:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:56 smithi066 ceph-mon[97920]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:25:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:56.314 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:25:56 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-0[112658]: 2023-12-11T19:25:56.183+0000 7f1e1a18f700 -1 osd.0 0 waiting for initial osdmap 2023-12-11T19:25:56.314 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:25:56 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-0[112658]: 2023-12-11T19:25:56.187+0000 7f1e14f87700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:25:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:56 smithi167 ceph-mon[102750]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-11T19:25:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:56 smithi167 ceph-mon[102750]: osdmap e6: 1 total, 0 up, 1 in 2023-12-11T19:25:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:56 smithi167 ceph-mon[102750]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:25:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:57 smithi167 ceph-mon[102750]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:25:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:57 smithi167 ceph-mon[102750]: osdmap e7: 1 total, 0 up, 1 in 2023-12-11T19:25:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:57 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:57 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:57 smithi167 ceph-mon[102750]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:57.534 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-11T19:25:57.534 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-11T19:25:57.534 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-12-11T19:25:57.534 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-12-11T19:25:57.534 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0301504 s, 348 MB/s 2023-12-11T19:25:57.534 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-12-11T19:25:57.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:57 smithi066 ceph-mon[97920]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:25:57.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:57 smithi066 ceph-mon[97920]: osdmap e7: 1 total, 0 up, 1 in 2023-12-11T19:25:57.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:57 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:57 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:57 smithi066 ceph-mon[97920]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:57.570 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:57 smithi070 ceph-mon[104044]: from='osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:25:57.571 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:57 smithi070 ceph-mon[104044]: osdmap e7: 1 total, 0 up, 1 in 2023-12-11T19:25:57.571 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:57 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.571 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:57 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:57.571 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:57 smithi070 ceph-mon[104044]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T19:25:58.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2023-12-11T19:25:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:58 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:25:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:58 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:25:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:58 smithi070 ceph-mon[104044]: osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] boot 2023-12-11T19:25:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:58 smithi070 ceph-mon[104044]: osdmap e8: 1 total, 1 up, 1 in 2023-12-11T19:25:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:58 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:58 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:25:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:58 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:25:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:58 smithi167 ceph-mon[102750]: osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] boot 2023-12-11T19:25:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:58 smithi167 ceph-mon[102750]: osdmap e8: 1 total, 1 up, 1 in 2023-12-11T19:25:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:58 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:58.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:58 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:25:58.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:58 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:25:58.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:58 smithi066 ceph-mon[97920]: osd.0 [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] boot 2023-12-11T19:25:58.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:58 smithi066 ceph-mon[97920]: osdmap e8: 1 total, 1 up, 1 in 2023-12-11T19:25:58.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:58 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T19:25:58.912 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: osdmap e9: 1 total, 1 up, 1 in 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi066 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi066 to 17271M 2023-12-11T19:25:59.462 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.463 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:59.463 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:59.463 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:25:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: osdmap e9: 1 total, 1 up, 1 in 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi066 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi066 to 17271M 2023-12-11T19:25:59.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:25:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: osdmap e9: 1 total, 1 up, 1 in 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi066 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:25:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi066 to 17271M 2023-12-11T19:25:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:25:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:25:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:25:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:25:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 -- 172.21.15.70:0/4038392675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 msgr2=0x7f418c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 --2- 172.21.15.70:0/4038392675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c101a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41800099c0 tx=0x7f4180009cd0 comp rx=0 tx=0).stop 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 -- 172.21.15.70:0/4038392675 shutdown_connections 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 --2- 172.21.15.70:0/4038392675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c101a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 --2- 172.21.15.70:0/4038392675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c1031c0 0x7f418c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 --2- 172.21.15.70:0/4038392675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c102820 0x7f418c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 -- 172.21.15.70:0/4038392675 >> 172.21.15.70:0/4038392675 conn(0x7f418c0fcbe0 msgr2=0x7f418c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 -- 172.21.15.70:0/4038392675 shutdown_connections 2023-12-11T19:26:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.446+0000 7f4192b6b700 1 -- 172.21.15.70:0/4038392675 wait complete. 2023-12-11T19:26:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.447+0000 7f4192b6b700 1 Processor -- start 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.447+0000 7f4192b6b700 1 -- start start 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4192b6b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4192b6b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c102820 0x7f418c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4192b6b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c1031c0 0x7f418c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4192b6b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c19b3b0 con 0x7f418c1031c0 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4192b6b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c074440 con 0x7f418c101620 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4192b6b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c074600 con 0x7f418c102820 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54942/0 (socket says 172.21.15.70:54942) 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 -- 172.21.15.70:0/115495059 learned_addr learned my addr 172.21.15.70:0/115495059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f418bfff700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c102820 0x7f418c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4191108700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c1031c0 0x7f418c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 -- 172.21.15.70:0/115495059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c102820 msgr2=0x7f418c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c102820 0x7f418c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 -- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c1031c0 msgr2=0x7f418c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c1031c0 0x7f418c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f4190907700 1 -- 172.21.15.70:0/115495059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800095f0 con 0x7f418c101620 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.448+0000 7f418bfff700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c102820 0x7f418c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.449+0000 7f4190907700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c075320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4180009f50 tx=0x7f4180003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.449+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418001d070 con 0x7f418c101620 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.449+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f418000bda0 con 0x7f418c101620 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.449+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c1a5d60 con 0x7f418c101620 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.449+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180022d60 con 0x7f418c101620 2023-12-11T19:26:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.449+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c1a60c0 con 0x7f418c101620 2023-12-11T19:26:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.450+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c108bf0 con 0x7f418c101620 2023-12-11T19:26:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.452+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f4180022500 con 0x7f418c101620 2023-12-11T19:26:00.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.453+0000 7f4189ffb700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f417c0a1470 0x7f417c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:00.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.454+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f41800cc8a0 con 0x7f418c101620 2023-12-11T19:26:00.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.454+0000 7f418bfff700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f417c0a1470 0x7f417c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:00.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.454+0000 7f418bfff700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f417c0a1470 0x7f417c0a3930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fd0 tx=0x7f4178005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:26:00.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.457+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4180097960 con 0x7f418c101620 2023-12-11T19:26:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:00.660+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f418c061a10 con 0x7f417c0a1470 2023-12-11T19:26:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:01.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:01.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:01.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:02.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:01 smithi167 ceph-mon[102750]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:02.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:01 smithi167 ceph-mon[102750]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:02.040 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:01 smithi070 ceph-mon[104044]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:02.041 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:01 smithi070 ceph-mon[104044]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:02.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:01 smithi066 ceph-mon[97920]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:02.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:01 smithi066 ceph-mon[97920]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:03 smithi167 ceph-mon[102750]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:04.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:03 smithi066 ceph-mon[97920]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:04.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:03 smithi070 ceph-mon[104044]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:05.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:04 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/2461992813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]: dispatch 2023-12-11T19:26:05.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:04 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]: dispatch 2023-12-11T19:26:05.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:04 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]': finished 2023-12-11T19:26:05.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:04 smithi167 ceph-mon[102750]: osdmap e10: 2 total, 1 up, 2 in 2023-12-11T19:26:05.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:04 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:05.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:04 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/2877572200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:26:05.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:04 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/2461992813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]: dispatch 2023-12-11T19:26:05.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:04 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]: dispatch 2023-12-11T19:26:05.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:04 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]': finished 2023-12-11T19:26:05.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:04 smithi066 ceph-mon[97920]: osdmap e10: 2 total, 1 up, 2 in 2023-12-11T19:26:05.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:04 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:05.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:04 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/2877572200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:26:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:04 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/2461992813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]: dispatch 2023-12-11T19:26:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:04 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]: dispatch 2023-12-11T19:26:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:04 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "820e245f-dede-4543-a91d-d7ba2568ff7d"}]': finished 2023-12-11T19:26:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:04 smithi070 ceph-mon[104044]: osdmap e10: 2 total, 1 up, 2 in 2023-12-11T19:26:05.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:04 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:05.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:04 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/2877572200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:26:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:05 smithi167 ceph-mon[102750]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:05 smithi066 ceph-mon[97920]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:06.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:05 smithi070 ceph-mon[104044]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:08.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:07 smithi070 ceph-mon[104044]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:08.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:07 smithi167 ceph-mon[102750]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:08.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:07 smithi066 ceph-mon[97920]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:10.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:09 smithi070 ceph-mon[104044]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:10.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:09 smithi167 ceph-mon[102750]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:09 smithi066 ceph-mon[97920]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:12.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:11 smithi167 ceph-mon[102750]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:12.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-11T19:26:12.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:12.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:11 smithi066 ceph-mon[97920]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:12.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-11T19:26:12.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:12.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:11 smithi070 ceph-mon[104044]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:12.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-11T19:26:12.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:13.161 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:12 smithi070 ceph-mon[104044]: Deploying daemon osd.1 on smithi070 2023-12-11T19:26:13.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:12 smithi167 ceph-mon[102750]: Deploying daemon osd.1 on smithi070 2023-12-11T19:26:13.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:12 smithi066 ceph-mon[97920]: Deploying daemon osd.1 on smithi070 2023-12-11T19:26:14.192 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:13 smithi070 ceph-mon[104044]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:14.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:13 smithi167 ceph-mon[102750]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:14.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:13 smithi066 ceph-mon[97920]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:16.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:15 smithi167 ceph-mon[102750]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:16.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:15 smithi066 ceph-mon[97920]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:16.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:15 smithi070 ceph-mon[104044]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:18.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:17 smithi167 ceph-mon[102750]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:18.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:17 smithi066 ceph-mon[97920]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:18.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:17 smithi070 ceph-mon[104044]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:19.283 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:19 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:19.283 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:19 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:19.283 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:19 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:19.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:19 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:19.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:19 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:19.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:19 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:19.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:19 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:19 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:19 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:20.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:20 smithi066 ceph-mon[97920]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:20.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:20 smithi070 ceph-mon[104044]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:20 smithi167 ceph-mon[102750]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:21.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.687+0000 7f4189ffb700 1 -- 172.21.15.70:0/115495059 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f418c061a10 con 0x7f417c0a1470 2023-12-11T19:26:21.689 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 1 on host 'smithi070' 2023-12-11T19:26:21.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.690+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f417c0a1470 msgr2=0x7f417c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:21.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f417c0a1470 0x7f417c0a3930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4178005fd0 tx=0x7f4178005f60 comp rx=0 tx=0).stop 2023-12-11T19:26:21.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 msgr2=0x7f418c075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c075320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4180009f50 tx=0x7f4180003a50 comp rx=0 tx=0).stop 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 shutdown_connections 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f417c0a1470 0x7f417c0a3930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f418c101620 0x7f418c075320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f418c1031c0 0x7f418c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 --2- 172.21.15.70:0/115495059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f418c102820 0x7f418c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 >> 172.21.15.70:0/115495059 conn(0x7f418c0fcbe0 msgr2=0x7f418c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 shutdown_connections 2023-12-11T19:26:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:26:21.691+0000 7f4192b6b700 1 -- 172.21.15.70:0/115495059 wait complete. 2023-12-11T19:26:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:21 smithi066 ceph-mon[97920]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:21.840 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:21 smithi070 ceph-mon[104044]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:21.840 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:21.840 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:21.841 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:21.841 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:21.841 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:22.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:21 smithi167 ceph-mon[102750]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:22.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:22.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:22.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:22.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:22.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:22.247 DEBUG:teuthology.orchestra.run.smithi070:osd.1> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.1.service 2023-12-11T19:26:22.251 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2023-12-11T19:26:22.251 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-11T19:26:22.326 INFO:journalctl@ceph.osd.1.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:26:22.929 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:22.929 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:22.929 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:22.929 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:22 smithi070 ceph-mon[104044]: from='osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T19:26:22.929 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:22 smithi070 ceph-mon[104044]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T19:26:23.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:23.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:23.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:23.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:22 smithi167 ceph-mon[102750]: from='osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T19:26:23.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:22 smithi167 ceph-mon[102750]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T19:26:23.029 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:26:23.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:23.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:23.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:23.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:22 smithi066 ceph-mon[97920]: from='osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T19:26:23.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:22 smithi066 ceph-mon[97920]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T19:26:23.841 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-mon[104044]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:23.842 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-mon[104044]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-11T19:26:23.842 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-mon[104044]: from='osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:26:23.842 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-mon[104044]: osdmap e11: 2 total, 1 up, 2 in 2023-12-11T19:26:23.842 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:23.842 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-mon[104044]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:26:23.842 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-1[109163]: 2023-12-11T19:26:23.707+0000 7fcbb6f6e700 -1 osd.1 0 waiting for initial osdmap 2023-12-11T19:26:23.842 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-1[109163]: 2023-12-11T19:26:23.712+0000 7fcbaf561700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:26:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:23 smithi167 ceph-mon[102750]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:23 smithi167 ceph-mon[102750]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-11T19:26:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:23 smithi167 ceph-mon[102750]: from='osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:26:23.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:23 smithi167 ceph-mon[102750]: osdmap e11: 2 total, 1 up, 2 in 2023-12-11T19:26:23.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:23 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:23.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:23 smithi167 ceph-mon[102750]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:26:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:23 smithi066 ceph-mon[97920]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:23 smithi066 ceph-mon[97920]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-11T19:26:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:23 smithi066 ceph-mon[97920]: from='osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:26:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:23 smithi066 ceph-mon[97920]: osdmap e11: 2 total, 1 up, 2 in 2023-12-11T19:26:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:23 smithi066 ceph-mon[97920]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:26:25.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:24 smithi167 ceph-mon[102750]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:26:25.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:24 smithi167 ceph-mon[102750]: osdmap e12: 2 total, 1 up, 2 in 2023-12-11T19:26:25.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:25.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:25.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:24 smithi066 ceph-mon[97920]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:26:25.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:24 smithi066 ceph-mon[97920]: osdmap e12: 2 total, 1 up, 2 in 2023-12-11T19:26:25.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:25.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:25.093 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-11T19:26:25.093 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-11T19:26:25.093 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-12-11T19:26:25.093 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-12-11T19:26:25.093 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0307162 s, 341 MB/s 2023-12-11T19:26:25.093 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-12-11T19:26:25.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:24 smithi070 ceph-mon[104044]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:26:25.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:24 smithi070 ceph-mon[104044]: osdmap e12: 2 total, 1 up, 2 in 2023-12-11T19:26:25.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:25.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:25.616 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2023-12-11T19:26:26.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:25 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:26:26.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:25 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:26:26.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:25 smithi167 ceph-mon[102750]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:26.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:25 smithi167 ceph-mon[102750]: osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] boot 2023-12-11T19:26:26.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:25 smithi167 ceph-mon[102750]: osdmap e13: 2 total, 2 up, 2 in 2023-12-11T19:26:26.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:25 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:26.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:25 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:26:26.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:25 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:26:26.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:25 smithi066 ceph-mon[97920]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:26.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:25 smithi066 ceph-mon[97920]: osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] boot 2023-12-11T19:26:26.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:25 smithi066 ceph-mon[97920]: osdmap e13: 2 total, 2 up, 2 in 2023-12-11T19:26:26.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:25 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:26.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:25 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:26:26.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:25 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:26:26.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:25 smithi070 ceph-mon[104044]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T19:26:26.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:25 smithi070 ceph-mon[104044]: osd.1 [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] boot 2023-12-11T19:26:26.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:25 smithi070 ceph-mon[104044]: osdmap e13: 2 total, 2 up, 2 in 2023-12-11T19:26:26.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:25 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T19:26:26.393 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:26:27.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi070 2023-12-11T19:26:27.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:26:27.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi070 to 17271M 2023-12-11T19:26:27.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:26 smithi167 ceph-mon[102750]: osdmap e14: 2 total, 2 up, 2 in 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi070 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi070 to 17271M 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:27.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:27.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:26 smithi066 ceph-mon[97920]: osdmap e14: 2 total, 2 up, 2 in 2023-12-11T19:26:27.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi070 2023-12-11T19:26:27.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:26:27.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi070 to 17271M 2023-12-11T19:26:27.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:27.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:27.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:27.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:26 smithi070 ceph-mon[104044]: osdmap e14: 2 total, 2 up, 2 in 2023-12-11T19:26:28.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 -- 172.21.15.167:0/3815014760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304101620 msgr2=0x7f9304101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:28.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 --2- 172.21.15.167:0/3815014760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304101620 0x7f9304101a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009990 tx=0x7f92f4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:26:28.020 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 -- 172.21.15.167:0/3815014760 shutdown_connections 2023-12-11T19:26:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 --2- 172.21.15.167:0/3815014760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93041031c0 0x7f9304109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 --2- 172.21.15.167:0/3815014760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304101620 0x7f9304101a40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 --2- 172.21.15.167:0/3815014760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9304102820 0x7f9304102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 -- 172.21.15.167:0/3815014760 >> 172.21.15.167:0/3815014760 conn(0x7f93040fcbe0 msgr2=0x7f93040ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:26:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 -- 172.21.15.167:0/3815014760 shutdown_connections 2023-12-11T19:26:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.019+0000 7f9308eba700 1 -- 172.21.15.167:0/3815014760 wait complete. 2023-12-11T19:26:28.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:27 smithi167 ceph-mon[102750]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:28.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.020+0000 7f9308eba700 1 Processor -- start 2023-12-11T19:26:28.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.020+0000 7f9308eba700 1 -- start start 2023-12-11T19:26:28.023 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9308eba700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9304101620 0x7f93041996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9308eba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 0x7f9304199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9308eba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93041031c0 0x7f93041a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9308eba700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a1220 con 0x7f9304101620 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9308eba700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a1360 con 0x7f93041031c0 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9308eba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a14f0 con 0x7f9304102820 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 0x7f9304199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 0x7f9304199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59438/0 (socket says 172.21.15.167:59438) 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 -- 172.21.15.167:0/3525720185 learned_addr learned my addr 172.21.15.167:0/3525720185 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9302d9d700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93041031c0 0x7f93041a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f930259c700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9304101620 0x7f93041996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 -- 172.21.15.167:0/3525720185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93041031c0 msgr2=0x7f93041a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93041031c0 0x7f93041a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 -- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9304101620 msgr2=0x7f93041996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9304101620 0x7f93041996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.021+0000 7f9301d9b700 1 -- 172.21.15.167:0/3525720185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40095f0 con 0x7f9304102820 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f930259c700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9304101620 0x7f93041996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f9301d9b700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 0x7f9304199c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9d0 tx=0x7f92f800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8009790 con 0x7f9304102820 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92f800cbd0 con 0x7f9304102820 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93041a17d0 con 0x7f9304102820 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8018680 con 0x7f9304102820 2023-12-11T19:26:28.025 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.022+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93041a1d20 con 0x7f9304102820 2023-12-11T19:26:28.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.023+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930404f290 con 0x7f9304102820 2023-12-11T19:26:28.026 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.025+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f92f800cd40 con 0x7f9304102820 2023-12-11T19:26:28.027 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.027+0000 7f92f37fe700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f92ec0a1470 0x7f92ec0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:28.027 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.027+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f92f8014070 con 0x7f9304102820 2023-12-11T19:26:28.028 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.027+0000 7f930259c700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f92ec0a1470 0x7f92ec0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:28.029 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.028+0000 7f930259c700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f92ec0a1470 0x7f92ec0a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b2e0 tx=0x7f92f4005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:26:28.030 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.030+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f92f80968e0 con 0x7f9304102820 2023-12-11T19:26:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:27 smithi066 ceph-mon[97920]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:27 smithi070 ceph-mon[104044]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:28.239 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:28.239+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9304109730 con 0x7f92ec0a1470 2023-12-11T19:26:29.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:28 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:29.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:28 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:29.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:28 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:29.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:28 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:29.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:28 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:29.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:28 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:29.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:28 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:29.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:28 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:29.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:28 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:29.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:29 smithi167 ceph-mon[102750]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:29.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:29 smithi167 ceph-mon[102750]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:30.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:29 smithi066 ceph-mon[97920]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:30.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:29 smithi066 ceph-mon[97920]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:30.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:29 smithi070 ceph-mon[104044]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:30.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:29 smithi070 ceph-mon[104044]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:31 smithi066 ceph-mon[97920]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:31 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/976976479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]: dispatch 2023-12-11T19:26:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:31 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]: dispatch 2023-12-11T19:26:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:31 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]': finished 2023-12-11T19:26:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:31 smithi066 ceph-mon[97920]: osdmap e15: 3 total, 2 up, 3 in 2023-12-11T19:26:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:31 smithi070 ceph-mon[104044]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:31 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/976976479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]: dispatch 2023-12-11T19:26:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:31 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]: dispatch 2023-12-11T19:26:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:31 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]': finished 2023-12-11T19:26:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:31 smithi070 ceph-mon[104044]: osdmap e15: 3 total, 2 up, 3 in 2023-12-11T19:26:32.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:31 smithi167 ceph-mon[102750]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:31 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/976976479' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]: dispatch 2023-12-11T19:26:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:31 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]: dispatch 2023-12-11T19:26:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:31 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cb5fd9c-e264-4947-85e5-413b230c4575"}]': finished 2023-12-11T19:26:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:31 smithi167 ceph-mon[102750]: osdmap e15: 3 total, 2 up, 3 in 2023-12-11T19:26:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:33.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:32 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/1568820923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:26:33.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:32 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/1568820923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:26:33.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:32 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/1568820923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:26:34.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:33 smithi066 ceph-mon[97920]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:34.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:33 smithi070 ceph-mon[104044]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:34.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:33 smithi167 ceph-mon[102750]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:36.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:35 smithi066 ceph-mon[97920]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:36.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:35 smithi070 ceph-mon[104044]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:36.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:35 smithi167 ceph-mon[102750]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:38.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:37 smithi167 ceph-mon[102750]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:38.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:37 smithi066 ceph-mon[97920]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:37 smithi070 ceph-mon[104044]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:40.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:39 smithi167 ceph-mon[102750]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:40.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:39 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-11T19:26:40.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:39 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:40.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:39 smithi167 ceph-mon[102750]: Deploying daemon osd.2 on smithi167 2023-12-11T19:26:40.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:39 smithi066 ceph-mon[97920]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:40.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:39 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-11T19:26:40.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:39 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:40.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:39 smithi066 ceph-mon[97920]: Deploying daemon osd.2 on smithi167 2023-12-11T19:26:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:39 smithi070 ceph-mon[104044]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:39 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-11T19:26:40.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:39 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:40.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:39 smithi070 ceph-mon[104044]: Deploying daemon osd.2 on smithi167 2023-12-11T19:26:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:41 smithi066 ceph-mon[97920]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:42.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:41 smithi070 ceph-mon[104044]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:42.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:41 smithi167 ceph-mon[102750]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:44.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:43 smithi066 ceph-mon[97920]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:43 smithi070 ceph-mon[104044]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:44.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:43 smithi167 ceph-mon[102750]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:45 smithi066 ceph-mon[97920]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:46.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:45 smithi070 ceph-mon[104044]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:45 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:45 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:45 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:46.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:45 smithi167 ceph-mon[102750]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:46.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:45 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:46.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:45 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:46.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:45 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:48.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:47 smithi167 ceph-mon[102750]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:48.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:47 smithi066 ceph-mon[97920]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:48.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:47 smithi070 ceph-mon[104044]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:49.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:49.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:49.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:49 smithi167 ceph-mon[102750]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:49.421 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.419+0000 7f92f37fe700 1 -- 172.21.15.167:0/3525720185 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9304109730 con 0x7f92ec0a1470 2023-12-11T19:26:49.422 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2023-12-11T19:26:49.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f92ec0a1470 msgr2=0x7f92ec0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:49.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f92ec0a1470 0x7f92ec0a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b2e0 tx=0x7f92f4005e80 comp rx=0 tx=0).stop 2023-12-11T19:26:49.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 msgr2=0x7f9304199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:49.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 0x7f9304199c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9d0 tx=0x7f92f800ece0 comp rx=0 tx=0).stop 2023-12-11T19:26:49.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 shutdown_connections 2023-12-11T19:26:49.424 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f92ec0a1470 0x7f92ec0a3930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:49.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93041031c0 0x7f93041a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:49.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9304101620 0x7f93041996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:49.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 --2- 172.21.15.167:0/3525720185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9304102820 0x7f9304199c20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:49.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 >> 172.21.15.167:0/3525720185 conn(0x7f93040fcbe0 msgr2=0x7f9304108010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:26:49.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 shutdown_connections 2023-12-11T19:26:49.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:26:49.422+0000 7f9308eba700 1 -- 172.21.15.167:0/3525720185 wait complete. 2023-12-11T19:26:49.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:49.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:49.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:49 smithi066 ceph-mon[97920]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:49.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:49.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:49.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:49.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:49 smithi070 ceph-mon[104044]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:49.920 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.2.service 2023-12-11T19:26:49.925 INFO:tasks.cephadm:Deploying osd.3 on smithi066 with /dev/vg_nvme/lv_3... 2023-12-11T19:26:49.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-11T19:26:50.002 INFO:journalctl@ceph.osd.2.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:26:50.649 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:50.649 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:50.649 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:50.649 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:50 smithi167 ceph-mon[102750]: from='osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T19:26:50.649 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:50 smithi167 ceph-mon[102750]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T19:26:50.733 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:26:50.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:50.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:50.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:50.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:50 smithi066 ceph-mon[97920]: from='osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T19:26:50.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:50 smithi066 ceph-mon[97920]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T19:26:50.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:50.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:50.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:26:50.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:50 smithi070 ceph-mon[104044]: from='osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T19:26:50.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:50 smithi070 ceph-mon[104044]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T19:26:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:51 smithi066 ceph-mon[97920]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:51 smithi066 ceph-mon[97920]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T19:26:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:51 smithi066 ceph-mon[97920]: from='osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:26:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:51 smithi066 ceph-mon[97920]: osdmap e16: 3 total, 2 up, 3 in 2023-12-11T19:26:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:51 smithi066 ceph-mon[97920]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:26:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-mon[102750]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-mon[102750]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T19:26:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-mon[102750]: from='osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:26:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-mon[102750]: osdmap e16: 3 total, 2 up, 3 in 2023-12-11T19:26:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-mon[102750]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:26:51.772 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-2[108594]: 2023-12-11T19:26:51.426+0000 7f494a31a700 -1 osd.2 0 waiting for initial osdmap 2023-12-11T19:26:51.772 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-2[108594]: 2023-12-11T19:26:51.431+0000 7f4946114700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:26:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:51 smithi070 ceph-mon[104044]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:51 smithi070 ceph-mon[104044]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T19:26:51.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:51 smithi070 ceph-mon[104044]: from='osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:26:51.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:51 smithi070 ceph-mon[104044]: osdmap e16: 3 total, 2 up, 3 in 2023-12-11T19:26:51.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:51.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:51 smithi070 ceph-mon[104044]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:26:52.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:52 smithi167 ceph-mon[102750]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:26:52.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:52 smithi167 ceph-mon[102750]: osdmap e17: 3 total, 2 up, 3 in 2023-12-11T19:26:52.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:52 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:52.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:52 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:52.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:52 smithi066 ceph-mon[97920]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:26:52.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:52 smithi066 ceph-mon[97920]: osdmap e17: 3 total, 2 up, 3 in 2023-12-11T19:26:52.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:52 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:52.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:52 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:52.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:52 smithi070 ceph-mon[104044]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:26:52.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:52 smithi070 ceph-mon[104044]: osdmap e17: 3 total, 2 up, 3 in 2023-12-11T19:26:52.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:52 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:52.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:52 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:52.861 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-11T19:26:52.861 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-11T19:26:52.862 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-12-11T19:26:52.862 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-12-11T19:26:52.862 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.041728 s, 251 MB/s 2023-12-11T19:26:52.862 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-12-11T19:26:53.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2023-12-11T19:26:53.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:26:53.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:26:53.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:53.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] boot 2023-12-11T19:26:53.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: osdmap e18: 3 total, 3 up, 3 in 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:53.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:53.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] boot 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: osdmap e18: 3 total, 3 up, 3 in 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:26:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:26:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T19:26:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: osd.2 [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] boot 2023-12-11T19:26:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: osdmap e18: 3 total, 3 up, 3 in 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:53.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:26:54.145 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:26:55.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:55 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi167 2023-12-11T19:26:55.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:55 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi167 to 17271M 2023-12-11T19:26:55.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:55 smithi066 ceph-mon[97920]: osdmap e19: 3 total, 3 up, 3 in 2023-12-11T19:26:55.034 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:55.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:55 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi167 2023-12-11T19:26:55.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:55 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi167 to 17271M 2023-12-11T19:26:55.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:55 smithi167 ceph-mon[102750]: osdmap e19: 3 total, 3 up, 3 in 2023-12-11T19:26:55.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:55.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:55 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi167 2023-12-11T19:26:55.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:55 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi167 to 17271M 2023-12-11T19:26:55.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:55 smithi070 ceph-mon[104044]: osdmap e19: 3 total, 3 up, 3 in 2023-12-11T19:26:55.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 -- 172.21.15.66:0/858146503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87281020a0 msgr2=0x7f8728104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 --2- 172.21.15.66:0/858146503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87281020a0 0x7f8728104490 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8724009990 tx=0x7f8724009ca0 comp rx=0 tx=0).stop 2023-12-11T19:26:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 -- 172.21.15.66:0/858146503 shutdown_connections 2023-12-11T19:26:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 --2- 172.21.15.66:0/858146503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8728107300 0x7f8728109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 --2- 172.21.15.66:0/858146503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87281020a0 0x7f8728104490 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 --2- 172.21.15.66:0/858146503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 -- 172.21.15.66:0/858146503 >> 172.21.15.66:0/858146503 conn(0x7f87280fbad0 msgr2=0x7f87280fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:26:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 -- 172.21.15.66:0/858146503 shutdown_connections 2023-12-11T19:26:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.676+0000 7f872fa57700 1 -- 172.21.15.66:0/858146503 wait complete. 2023-12-11T19:26:55.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 Processor -- start 2023-12-11T19:26:55.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 -- start start 2023-12-11T19:26:55.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87281020a0 0x7f8728197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:55.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:55.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8728107300 0x7f872819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:55.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872819ef90 con 0x7f8728107300 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872819f0d0 con 0x7f87281049d0 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.677+0000 7f872fa57700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872819f260 con 0x7f87281020a0 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872d7f3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87281020a0 0x7f8728197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872dff4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8728107300 0x7f872819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:59448/0 (socket says 172.21.15.66:59448) 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 -- 172.21.15.66:0/2960618462 learned_addr learned my addr 172.21.15.66:0/2960618462 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:26:55.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 -- 172.21.15.66:0/2960618462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87281020a0 msgr2=0x7f8728197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87281020a0 0x7f8728197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 -- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8728107300 msgr2=0x7f872819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8728107300 0x7f872819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872cff2700 1 -- 172.21.15.66:0/2960618462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240095f0 con 0x7f87281049d0 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.678+0000 7f872dff4700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8728107300 0x7f872819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.679+0000 7f872cff2700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728197990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f871800d950 tx=0x7f871800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.679+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871800f890 con 0x7f87281049d0 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.679+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871800fe80 con 0x7f87281049d0 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.679+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872819f540 con 0x7f87281049d0 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.679+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871800e3b0 con 0x7f87281049d0 2023-12-11T19:26:55.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.679+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872819fa40 con 0x7f87281049d0 2023-12-11T19:26:55.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.680+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87280fd650 con 0x7f87281049d0 2023-12-11T19:26:55.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.682+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f8718010460 con 0x7f87281049d0 2023-12-11T19:26:55.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.684+0000 7f871e7fc700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f87140a13f0 0x7f87140a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:26:55.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.684+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f87180cb680 con 0x7f87281049d0 2023-12-11T19:26:55.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.684+0000 7f872d7f3700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f87140a13f0 0x7f87140a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:26:55.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.685+0000 7f872d7f3700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f87140a13f0 0x7f87140a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8724000c00 tx=0x7f8724005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:26:55.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.687+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8718096100 con 0x7f87281049d0 2023-12-11T19:26:55.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:55.909+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f872802d380 con 0x7f87140a13f0 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: pgmap v91: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: osdmap e20: 3 total, 3 up, 3 in 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: pgmap v91: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:26:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-11T19:26:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-11T19:26:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: osdmap e20: 3 total, 3 up, 3 in 2023-12-11T19:26:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T19:26:56.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:56.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:56.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:56.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: pgmap v91: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: osdmap e20: 3 total, 3 up, 3 in 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:26:56.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:26:57.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:26:57.086+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f87180964d0 con 0x7f87281049d0 2023-12-11T19:26:57.242 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:57 smithi066 ceph-mon[97920]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:57.243 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:57 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-11T19:26:57.243 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:57 smithi066 ceph-mon[97920]: osdmap e21: 3 total, 3 up, 3 in 2023-12-11T19:26:57.243 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:56 smithi066 sudo[118679]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T19:26:57.243 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:56 smithi066 sudo[118679]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:26:57.243 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:56 smithi066 sudo[118679]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:26:57.243 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:56 smithi066 sudo[118679]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:26:57.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:57 smithi070 ceph-mon[104044]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:57.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:57 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-11T19:26:57.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:57 smithi070 ceph-mon[104044]: osdmap e21: 3 total, 3 up, 3 in 2023-12-11T19:26:57.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:57 smithi167 ceph-mon[102750]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:26:57.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:57 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-11T19:26:57.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:57 smithi167 ceph-mon[102750]: osdmap e21: 3 total, 3 up, 3 in 2023-12-11T19:26:57.982 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:57 smithi066 sudo[118733]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T19:26:57.983 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:57 smithi066 sudo[118733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:26:57.983 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:57 smithi066 sudo[118733]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:26:57.983 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:26:57 smithi066 sudo[118733]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:26:58.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:58 smithi066 ceph-mon[97920]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:26:58.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:58 smithi066 ceph-mon[97920]: osdmap e22: 3 total, 3 up, 3 in 2023-12-11T19:26:58.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:58 smithi066 ceph-mon[97920]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-11T19:26:58.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:58 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:58 smithi070 ceph-mon[104044]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:26:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:58 smithi070 ceph-mon[104044]: osdmap e22: 3 total, 3 up, 3 in 2023-12-11T19:26:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:58 smithi070 ceph-mon[104044]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-11T19:26:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:58 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:58.361 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113954]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T19:26:58.361 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113954]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:26:58.361 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113954]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:26:58.361 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113954]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:26:58.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:58 smithi167 ceph-mon[102750]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:26:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:58 smithi167 ceph-mon[102750]: osdmap e22: 3 total, 3 up, 3 in 2023-12-11T19:26:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:58 smithi167 ceph-mon[102750]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-11T19:26:58.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:58 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:26:59.110 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113957]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T19:26:59.110 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113957]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:26:59.110 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113957]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:26:59.110 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:26:58 smithi070 sudo[113957]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:26:59.771 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113706]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T19:26:59.771 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113706]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:26:59.771 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113706]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:26:59.771 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113706]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:59 smithi066 ceph-mon[97920]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:59 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3882633567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2c58a66-528c-490e-8f4d-6fdb5be75536"}]: dispatch 2023-12-11T19:27:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:59 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3882633567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2c58a66-528c-490e-8f4d-6fdb5be75536"}]': finished 2023-12-11T19:27:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:59 smithi066 ceph-mon[97920]: osdmap e23: 4 total, 3 up, 4 in 2023-12-11T19:27:00.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:26:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:59 smithi070 ceph-mon[104044]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:59 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3882633567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2c58a66-528c-490e-8f4d-6fdb5be75536"}]: dispatch 2023-12-11T19:27:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:59 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3882633567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2c58a66-528c-490e-8f4d-6fdb5be75536"}]': finished 2023-12-11T19:27:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:59 smithi070 ceph-mon[104044]: osdmap e23: 4 total, 3 up, 4 in 2023-12-11T19:27:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:26:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:00.270 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113709]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T19:27:00.271 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113709]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:00.271 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113709]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:00.271 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:26:59 smithi167 sudo[113709]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:59 smithi167 ceph-mon[102750]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3882633567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2c58a66-528c-490e-8f4d-6fdb5be75536"}]: dispatch 2023-12-11T19:27:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3882633567' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2c58a66-528c-490e-8f4d-6fdb5be75536"}]': finished 2023-12-11T19:27:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:59 smithi167 ceph-mon[102750]: osdmap e23: 4 total, 3 up, 4 in 2023-12-11T19:27:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:26:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:00 smithi066 sudo[119100]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T19:27:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:00 smithi066 sudo[119100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:00 smithi066 sudo[119100]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:01.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:00 smithi066 sudo[119100]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:01.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:01 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:01.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:01 smithi167 ceph-mon[102750]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:01 smithi066 sudo[119477]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T19:27:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:01 smithi066 sudo[119477]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:01 smithi066 sudo[119477]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:01 smithi066 sudo[119477]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:01.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:01 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:01.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:01 smithi066 ceph-mon[97920]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:01.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:01 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:01.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:01 smithi070 ceph-mon[104044]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:02.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:01 smithi167 sudo[113720]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T19:27:02.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:01 smithi167 sudo[113720]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:01 smithi167 sudo[113720]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:01 smithi167 sudo[113720]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 sudo[113723]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 sudo[113723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 sudo[113723]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 sudo[113723]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2881597178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:02.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2881597178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:02.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:03.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:02 smithi070 sudo[113960]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T19:27:03.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:02 smithi070 sudo[113960]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:03.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:02 smithi070 sudo[113960]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:03.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:02 smithi070 sudo[113960]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:03.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:03 smithi167 ceph-mon[102750]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:03.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:03 smithi167 ceph-mon[102750]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:03.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:03 smithi167 ceph-mon[102750]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:03.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:03.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:03.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:03 smithi066 ceph-mon[97920]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:03 smithi066 ceph-mon[97920]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:03 smithi066 ceph-mon[97920]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:03.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 sudo[113963]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 sudo[113963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 sudo[113963]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 sudo[113963]: pam_unix(sudo:session): session closed for user root 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2881597178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:03.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:03.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:03.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:03.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T19:27:03.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T19:27:03.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T19:27:04.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:04 smithi167 ceph-mon[102750]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:04.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:04 smithi167 ceph-mon[102750]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:04.807 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:04 smithi066 ceph-mon[97920]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:04.807 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:04 smithi066 ceph-mon[97920]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:04.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:04 smithi070 ceph-mon[104044]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T19:27:04.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:04 smithi070 ceph-mon[104044]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T19:27:05.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:05 smithi167 ceph-mon[102750]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:05.788 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:05 smithi066 ceph-mon[97920]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:05.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:05 smithi070 ceph-mon[104044]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:07.758 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:07 smithi066 ceph-mon[97920]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-12-11T19:27:07.758 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-11T19:27:07.759 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:07.759 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:07 smithi066 ceph-mon[97920]: Deploying daemon osd.3 on smithi066 2023-12-11T19:27:07.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:07 smithi167 ceph-mon[102750]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-12-11T19:27:07.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-11T19:27:07.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:07 smithi167 ceph-mon[102750]: Deploying daemon osd.3 on smithi066 2023-12-11T19:27:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:07 smithi070 ceph-mon[104044]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-12-11T19:27:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-11T19:27:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:07.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:07 smithi070 ceph-mon[104044]: Deploying daemon osd.3 on smithi066 2023-12-11T19:27:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:09 smithi167 ceph-mon[102750]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-12-11T19:27:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:09 smithi066 ceph-mon[97920]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-12-11T19:27:09.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:09 smithi070 ceph-mon[104044]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-12-11T19:27:11.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:11 smithi167 ceph-mon[102750]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-12-11T19:27:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:11 smithi066 ceph-mon[97920]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-12-11T19:27:11.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:11 smithi070 ceph-mon[104044]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-12-11T19:27:14.333 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:14 smithi066 ceph-mon[97920]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-11T19:27:14.333 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:14.333 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:14.333 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:14 smithi070 ceph-mon[104044]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-11T19:27:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:14 smithi167 ceph-mon[102750]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-11T19:27:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:16.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:16 smithi070 ceph-mon[104044]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-11T19:27:16.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:16.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:16.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:16 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:16 smithi167 ceph-mon[102750]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-11T19:27:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:16 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:16 smithi066 ceph-mon[97920]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-11T19:27:16.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:16.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:16.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:16 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:16.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.571+0000 7f871e7fc700 1 -- 172.21.15.66:0/2960618462 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f872802d380 con 0x7f87140a13f0 2023-12-11T19:27:16.573 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 3 on host 'smithi066' 2023-12-11T19:27:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f87140a13f0 msgr2=0x7f87140a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f87140a13f0 0x7f87140a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8724000c00 tx=0x7f8724005fb0 comp rx=0 tx=0).stop 2023-12-11T19:27:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 msgr2=0x7f8728197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728197990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f871800d950 tx=0x7f871800dc60 comp rx=0 tx=0).stop 2023-12-11T19:27:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 shutdown_connections 2023-12-11T19:27:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f87140a13f0 0x7f87140a38b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8728107300 0x7f872819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87281020a0 0x7f8728197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 --2- 172.21.15.66:0/2960618462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87281049d0 0x7f8728197990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 >> 172.21.15.66:0/2960618462 conn(0x7f87280fbad0 msgr2=0x7f8728103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:27:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 shutdown_connections 2023-12-11T19:27:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:27:16.574+0000 7f872fa57700 1 -- 172.21.15.66:0/2960618462 wait complete. 2023-12-11T19:27:16.992 DEBUG:teuthology.orchestra.run.smithi066:osd.3> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.3.service 2023-12-11T19:27:16.996 INFO:tasks.cephadm:Deploying osd.4 on smithi070 with /dev/vg_nvme/lv_3... 2023-12-11T19:27:16.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-11T19:27:17.100 INFO:journalctl@ceph.osd.3.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:27:17.313 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:27:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:27:17.108+0000 7f351241c640 -1 osd.3 0 log_to_monitors true 2023-12-11T19:27:17.792 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:27:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:17 smithi066 ceph-mon[97920]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-11T19:27:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:17 smithi066 ceph-mon[97920]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-11T19:27:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:17 smithi070 ceph-mon[104044]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-11T19:27:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:17 smithi070 ceph-mon[104044]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-11T19:27:18.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:17 smithi167 ceph-mon[102750]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-11T19:27:18.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:18.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:18.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:18.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:17 smithi167 ceph-mon[102750]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-11T19:27:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:19 smithi070 ceph-mon[104044]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-11T19:27:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:19 smithi070 ceph-mon[104044]: osdmap e24: 4 total, 3 up, 4 in 2023-12-11T19:27:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:19 smithi070 ceph-mon[104044]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:27:19.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:19 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:19 smithi167 ceph-mon[102750]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-11T19:27:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:19 smithi167 ceph-mon[102750]: osdmap e24: 4 total, 3 up, 4 in 2023-12-11T19:27:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:19 smithi167 ceph-mon[102750]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:27:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:19 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:19.563 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:27:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:27:19.110+0000 7f3507a93700 -1 osd.3 0 waiting for initial osdmap 2023-12-11T19:27:19.563 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:27:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:27:19.116+0000 7f350308c700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:27:19.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:19 smithi066 ceph-mon[97920]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-11T19:27:19.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:19 smithi066 ceph-mon[97920]: osdmap e24: 4 total, 3 up, 4 in 2023-12-11T19:27:19.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:19 smithi066 ceph-mon[97920]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:27:19.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:19 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-11T19:27:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-11T19:27:19.855 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-12-11T19:27:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-12-11T19:27:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0384676 s, 273 MB/s 2023-12-11T19:27:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-12-11T19:27:20.248 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:20.248 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:27:20.248 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: osdmap e25: 4 total, 3 up, 4 in 2023-12-11T19:27:20.249 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.249 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.249 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] boot 2023-12-11T19:27:20.249 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: osdmap e26: 4 total, 4 up, 4 in 2023-12-11T19:27:20.249 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: osdmap e25: 4 total, 3 up, 4 in 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] boot 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: osdmap e26: 4 total, 4 up, 4 in 2023-12-11T19:27:20.422 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: from='osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: osdmap e25: 4 total, 3 up, 4 in 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: osd.3 [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] boot 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: osdmap e26: 4 total, 4 up, 4 in 2023-12-11T19:27:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T19:27:21.058 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:21.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:21 smithi070 ceph-mon[104044]: osdmap e27: 4 total, 4 up, 4 in 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:21 smithi167 ceph-mon[102750]: osdmap e27: 4 total, 4 up, 4 in 2023-12-11T19:27:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:27:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:27:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:21.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:21 smithi066 ceph-mon[97920]: osdmap e27: 4 total, 4 up, 4 in 2023-12-11T19:27:22.333 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:22 smithi070 ceph-mon[104044]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:22.333 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:22 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi066 2023-12-11T19:27:22.333 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:22 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi066 to 8635M 2023-12-11T19:27:22.333 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:22 smithi070 ceph-mon[104044]: osdmap e28: 4 total, 4 up, 4 in 2023-12-11T19:27:22.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:22 smithi167 ceph-mon[102750]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:22.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:22 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi066 2023-12-11T19:27:22.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:22 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi066 to 8635M 2023-12-11T19:27:22.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:22 smithi167 ceph-mon[102750]: osdmap e28: 4 total, 4 up, 4 in 2023-12-11T19:27:22.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:22 smithi066 ceph-mon[97920]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:22.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:22 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi066 2023-12-11T19:27:22.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:22 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi066 to 8635M 2023-12-11T19:27:22.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:22 smithi066 ceph-mon[97920]: osdmap e28: 4 total, 4 up, 4 in 2023-12-11T19:27:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 -- 172.21.15.70:0/4054681826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c941094f0 msgr2=0x7f2c9410b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 --2- 172.21.15.70:0/4054681826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c941094f0 0x7f2c9410b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b0a0 tx=0x7f2c8c00b3b0 comp rx=0 tx=0).stop 2023-12-11T19:27:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 -- 172.21.15.70:0/4054681826 shutdown_connections 2023-12-11T19:27:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 --2- 172.21.15.70:0/4054681826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c941094f0 0x7f2c9410b980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 --2- 172.21.15.70:0/4054681826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c94075600 0x7f2c94073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 --2- 172.21.15.70:0/4054681826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 0x7f2c940744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 -- 172.21.15.70:0/4054681826 >> 172.21.15.70:0/4054681826 conn(0x7f2c940fcbe0 msgr2=0x7f2c940ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:27:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 -- 172.21.15.70:0/4054681826 shutdown_connections 2023-12-11T19:27:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.588+0000 7f2c9add7700 1 -- 172.21.15.70:0/4054681826 wait complete. 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.589+0000 7f2c9add7700 1 Processor -- start 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.589+0000 7f2c9add7700 1 -- start start 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c9add7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 0x7f2c94199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c9add7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 0x7f2c94199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c9add7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c941094f0 0x7f2c941a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c9add7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c941a13b0 con 0x7f2c94074030 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c9add7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c941a14f0 con 0x7f2c94075600 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c9add7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c941a1680 con 0x7f2c941094f0 2023-12-11T19:27:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 0x7f2c94199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 0x7f2c94199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35900/0 (socket says 172.21.15.70:35900) 2023-12-11T19:27:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 -- 172.21.15.70:0/1096621903 learned_addr learned my addr 172.21.15.70:0/1096621903 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-12-11T19:27:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c98b73700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 0x7f2c94199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c99374700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c941094f0 0x7f2c941a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 -- 172.21.15.70:0/1096621903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c941094f0 msgr2=0x7f2c941a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c941094f0 0x7f2c941a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 -- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 msgr2=0x7f2c94199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 0x7f2c94199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c93fff700 1 -- 172.21.15.70:0/1096621903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c009d00 con 0x7f2c94075600 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.590+0000 7f2c98b73700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 0x7f2c94199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.591+0000 7f2c93fff700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 0x7f2c94199db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b500 tx=0x7f2c8000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.591+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011830 con 0x7f2c94075600 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.591+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c80011e20 con 0x7f2c94075600 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.591+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80010e00 con 0x7f2c94075600 2023-12-11T19:27:22.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.591+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c941a1960 con 0x7f2c94075600 2023-12-11T19:27:22.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.591+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c941a1e10 con 0x7f2c94075600 2023-12-11T19:27:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.592+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c941082b0 con 0x7f2c94075600 2023-12-11T19:27:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.594+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2c80011990 con 0x7f2c94075600 2023-12-11T19:27:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.595+0000 7f2c91ffb700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c840a1480 0x7f2c840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.596+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f2c800cb5e0 con 0x7f2c94075600 2023-12-11T19:27:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.596+0000 7f2c98b73700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c840a1480 0x7f2c840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.597+0000 7f2c98b73700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c840a1480 0x7f2c840a3940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005950 tx=0x7f2c880058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:27:22.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.600+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2c80095da0 con 0x7f2c94075600 2023-12-11T19:27:22.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:22.808+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c9402d420 con 0x7f2c840a1480 2023-12-11T19:27:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:23 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:27:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:23 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:27:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:23 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:23.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:27:23.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:27:23.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:23 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:23.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:23 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:27:23.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:23 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:27:23.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:23 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:24.242 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:24 smithi070 ceph-mon[104044]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:24.242 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:24 smithi070 ceph-mon[104044]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:27:24.242 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:24 smithi070 ceph-mon[104044]: osdmap e29: 4 total, 4 up, 4 in 2023-12-11T19:27:24.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:24 smithi167 ceph-mon[102750]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:24.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:24 smithi167 ceph-mon[102750]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:27:24.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:24 smithi167 ceph-mon[102750]: osdmap e29: 4 total, 4 up, 4 in 2023-12-11T19:27:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:24 smithi066 ceph-mon[97920]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:24 smithi066 ceph-mon[97920]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:27:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:24 smithi066 ceph-mon[97920]: osdmap e29: 4 total, 4 up, 4 in 2023-12-11T19:27:25.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:25 smithi070 ceph-mon[104044]: osdmap e30: 4 total, 4 up, 4 in 2023-12-11T19:27:25.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:25 smithi167 ceph-mon[102750]: osdmap e30: 4 total, 4 up, 4 in 2023-12-11T19:27:25.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:25 smithi066 ceph-mon[97920]: osdmap e30: 4 total, 4 up, 4 in 2023-12-11T19:27:26.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:26 smithi167 ceph-mon[102750]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:26.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:26 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/2926914591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]: dispatch 2023-12-11T19:27:26.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:26 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]: dispatch 2023-12-11T19:27:26.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:26 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]': finished 2023-12-11T19:27:26.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:26 smithi167 ceph-mon[102750]: osdmap e31: 5 total, 4 up, 5 in 2023-12-11T19:27:26.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:26 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:26.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:26 smithi066 ceph-mon[97920]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:26.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:26 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/2926914591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]: dispatch 2023-12-11T19:27:26.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:26 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]: dispatch 2023-12-11T19:27:26.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:26 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]': finished 2023-12-11T19:27:26.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:26 smithi066 ceph-mon[97920]: osdmap e31: 5 total, 4 up, 5 in 2023-12-11T19:27:26.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:26 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:26.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:26 smithi070 ceph-mon[104044]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:26.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:26 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/2926914591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]: dispatch 2023-12-11T19:27:26.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:26 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]: dispatch 2023-12-11T19:27:26.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:26 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "181213ac-13a3-4539-9435-df0f7299186b"}]': finished 2023-12-11T19:27:26.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:26 smithi070 ceph-mon[104044]: osdmap e31: 5 total, 4 up, 5 in 2023-12-11T19:27:26.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:26 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:27 smithi167 ceph-mon[102750]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2023-12-11T19:27:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:27 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/3431246978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:27 smithi066 ceph-mon[97920]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2023-12-11T19:27:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:27 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/3431246978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:27.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:27 smithi070 ceph-mon[104044]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2023-12-11T19:27:27.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:27 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/3431246978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:29.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:29 smithi167 ceph-mon[102750]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:27:29.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:29 smithi066 ceph-mon[97920]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:27:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:29 smithi070 ceph-mon[104044]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:27:31.750 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:31 smithi070 ceph-mon[104044]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-12-11T19:27:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:31 smithi167 ceph-mon[102750]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-12-11T19:27:31.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:31 smithi066 ceph-mon[97920]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-12-11T19:27:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:33 smithi167 ceph-mon[102750]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-12-11T19:27:33.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:33 smithi066 ceph-mon[97920]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-12-11T19:27:33.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:33 smithi070 ceph-mon[104044]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-12-11T19:27:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-11T19:27:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:34 smithi167 ceph-mon[102750]: Deploying daemon osd.4 on smithi070 2023-12-11T19:27:34.786 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-11T19:27:34.786 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:34.786 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:34 smithi070 ceph-mon[104044]: Deploying daemon osd.4 on smithi070 2023-12-11T19:27:34.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-11T19:27:34.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:34.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:34 smithi066 ceph-mon[97920]: Deploying daemon osd.4 on smithi070 2023-12-11T19:27:35.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:35 smithi167 ceph-mon[102750]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-11T19:27:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:35 smithi066 ceph-mon[97920]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-11T19:27:35.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:35 smithi070 ceph-mon[104044]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-11T19:27:37.752 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:37 smithi070 ceph-mon[104044]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:37.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:37 smithi167 ceph-mon[102750]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:37 smithi066 ceph-mon[97920]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:39.737 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:39 smithi070 ceph-mon[104044]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:39.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:39 smithi167 ceph-mon[102750]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:39.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:39 smithi066 ceph-mon[97920]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:41.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:41.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:41.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:41.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:41.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:41.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:41.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:42.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:41 smithi070 ceph-mon[104044]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:42.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:41 smithi167 ceph-mon[102750]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:42.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:41 smithi066 ceph-mon[97920]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:43.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.438+0000 7f2c91ffb700 1 -- 172.21.15.70:0/1096621903 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c9402d420 con 0x7f2c840a1480 2023-12-11T19:27:43.440 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 4 on host 'smithi070' 2023-12-11T19:27:43.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c840a1480 msgr2=0x7f2c840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:43.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c840a1480 0x7f2c840a3940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005950 tx=0x7f2c880058e0 comp rx=0 tx=0).stop 2023-12-11T19:27:43.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 msgr2=0x7f2c94199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:43.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 0x7f2c94199db0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b500 tx=0x7f2c8000b810 comp rx=0 tx=0).stop 2023-12-11T19:27:43.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 shutdown_connections 2023-12-11T19:27:43.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c840a1480 0x7f2c840a3940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:43.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c941094f0 0x7f2c941a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:43.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c94074030 0x7f2c94199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:43.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 --2- 172.21.15.70:0/1096621903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c94075600 0x7f2c94199db0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:43.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 >> 172.21.15.70:0/1096621903 conn(0x7f2c940fcbe0 msgr2=0x7f2c941053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:27:43.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 shutdown_connections 2023-12-11T19:27:43.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:27:43.440+0000 7f2c9add7700 1 -- 172.21.15.70:0/1096621903 wait complete. 2023-12-11T19:27:43.899 DEBUG:teuthology.orchestra.run.smithi070:osd.4> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.4.service 2023-12-11T19:27:43.905 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2023-12-11T19:27:43.906 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-11T19:27:43.987 INFO:journalctl@ceph.osd.4.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:27:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:44.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:44.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:44.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:27:44.705 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:27:44.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:44 smithi070 ceph-mon[104044]: from='osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T19:27:44.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:44 smithi070 ceph-mon[104044]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T19:27:45.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:44 smithi167 ceph-mon[102750]: from='osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T19:27:45.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:44 smithi167 ceph-mon[102750]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T19:27:45.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:44 smithi066 ceph-mon[97920]: from='osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T19:27:45.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:44 smithi066 ceph-mon[97920]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T19:27:45.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:45 smithi167 ceph-mon[102750]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:45.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:45 smithi167 ceph-mon[102750]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-11T19:27:45.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:45 smithi167 ceph-mon[102750]: from='osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:27:45.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:45 smithi167 ceph-mon[102750]: osdmap e32: 5 total, 4 up, 5 in 2023-12-11T19:27:45.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:45 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:45.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:45 smithi167 ceph-mon[102750]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:27:46.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-mon[104044]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:46.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-mon[104044]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-11T19:27:46.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-mon[104044]: from='osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:27:46.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-mon[104044]: osdmap e32: 5 total, 4 up, 5 in 2023-12-11T19:27:46.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:46.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-mon[104044]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:27:46.112 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:27:45.845+0000 7f8658567700 -1 osd.4 0 waiting for initial osdmap 2023-12-11T19:27:46.112 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:27:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:27:45.853+0000 7f8652b5e700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:27:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:45 smithi066 ceph-mon[97920]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:45 smithi066 ceph-mon[97920]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-11T19:27:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:45 smithi066 ceph-mon[97920]: from='osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:27:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:45 smithi066 ceph-mon[97920]: osdmap e32: 5 total, 4 up, 5 in 2023-12-11T19:27:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:45 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:46.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:45 smithi066 ceph-mon[97920]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:27:46.773 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-11T19:27:46.773 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-11T19:27:46.773 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-12-11T19:27:46.773 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-12-11T19:27:46.774 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0311031 s, 337 MB/s 2023-12-11T19:27:46.774 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-12-11T19:27:47.095 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:46 smithi070 ceph-mon[104044]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:27:47.095 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:46 smithi070 ceph-mon[104044]: osdmap e33: 5 total, 4 up, 5 in 2023-12-11T19:27:47.095 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:47.095 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:47.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:46 smithi167 ceph-mon[102750]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:27:47.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:46 smithi167 ceph-mon[102750]: osdmap e33: 5 total, 4 up, 5 in 2023-12-11T19:27:47.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:47.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:47.195 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2023-12-11T19:27:47.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:46 smithi066 ceph-mon[97920]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:27:47.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:46 smithi066 ceph-mon[97920]: osdmap e33: 5 total, 4 up, 5 in 2023-12-11T19:27:47.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:47.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:47.980 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] boot 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: osdmap e34: 5 total, 5 up, 5 in 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:48.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:48.113 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:27:48.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:27:48.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:48.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:48.264 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] boot 2023-12-11T19:27:48.264 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: osdmap e34: 5 total, 5 up, 5 in 2023-12-11T19:27:48.264 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:48.264 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.264 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:48.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:48.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:48.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:48.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: osd.4 [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] boot 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: osdmap e34: 5 total, 5 up, 5 in 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:48.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:48.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:27:48.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:27:49.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:48 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi070 2023-12-11T19:27:49.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:48 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi070 to 8635M 2023-12-11T19:27:49.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:48 smithi070 ceph-mon[104044]: osdmap e35: 5 total, 5 up, 5 in 2023-12-11T19:27:49.259 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:48 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi070 2023-12-11T19:27:49.259 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:48 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi070 to 8635M 2023-12-11T19:27:49.259 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:48 smithi167 ceph-mon[102750]: osdmap e35: 5 total, 5 up, 5 in 2023-12-11T19:27:49.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:48 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi070 2023-12-11T19:27:49.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:48 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi070 to 8635M 2023-12-11T19:27:49.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:48 smithi066 ceph-mon[97920]: osdmap e35: 5 total, 5 up, 5 in 2023-12-11T19:27:49.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 -- 172.21.15.167:0/2709189040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd980102820 msgr2=0x7fd980102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:49.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 --2- 172.21.15.167:0/2709189040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd980102820 0x7fd980102c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd974009990 tx=0x7fd974009ca0 comp rx=0 tx=0).stop 2023-12-11T19:27:49.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 -- 172.21.15.167:0/2709189040 shutdown_connections 2023-12-11T19:27:49.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 --2- 172.21.15.167:0/2709189040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980101620 0x7fd980101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:49.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 --2- 172.21.15.167:0/2709189040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9801031c0 0x7fd980109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:49.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 --2- 172.21.15.167:0/2709189040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd980102820 0x7fd980102c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:49.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 -- 172.21.15.167:0/2709189040 >> 172.21.15.167:0/2709189040 conn(0x7fd9800fcbe0 msgr2=0x7fd9800ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:27:49.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 -- 172.21.15.167:0/2709189040 shutdown_connections 2023-12-11T19:27:49.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.506+0000 7fd9858b0700 1 -- 172.21.15.167:0/2709189040 wait complete. 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.507+0000 7fd9858b0700 1 Processor -- start 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.507+0000 7fd9858b0700 1 -- start start 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd9858b0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd980101620 0x7fd980199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd9858b0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980102820 0x7fd980199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd9858b0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 0x7fd9801a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd9858b0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9801a1270 con 0x7fd980101620 2023-12-11T19:27:49.517 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd9858b0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9801a13b0 con 0x7fd980102820 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd9858b0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9801a1540 con 0x7fd9801031c0 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 0x7fd9801a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 0x7fd9801a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60620/0 (socket says 172.21.15.167:60620) 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 -- 172.21.15.167:0/3031409793 learned_addr learned my addr 172.21.15.167:0/3031409793 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97effd700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd980101620 0x7fd980199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97e7fc700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980102820 0x7fd980199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 -- 172.21.15.167:0/3031409793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980102820 msgr2=0x7fd980199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980102820 0x7fd980199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 -- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd980101620 msgr2=0x7fd980199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd980101620 0x7fd980199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97f7fe700 1 -- 172.21.15.167:0/3031409793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740095f0 con 0x7fd9801031c0 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.508+0000 7fd97e7fc700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980102820 0x7fd980199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:27:49.518 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.509+0000 7fd97f7fe700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 0x7fd9801a0ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd978007f10 tx=0x7fd97800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.509+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800ddd0 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.509+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd97800f040 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.509+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978004870 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.509+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9801a1820 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.509+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9801a1cd0 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.510+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98004f290 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.512+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd978003a00 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.514+0000 7fd9848ae700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd9680a1480 0x7fd9680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.515+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fd9780cb720 con 0x7fd9801031c0 2023-12-11T19:27:49.519 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.515+0000 7fd97effd700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd9680a1480 0x7fd9680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:27:49.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.516+0000 7fd97effd700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd9680a1480 0x7fd9680a3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd970005950 tx=0x7fd9700058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:27:49.520 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.519+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd97804f2f0 con 0x7fd9801031c0 2023-12-11T19:27:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:27:49.725+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd980109780 con 0x7fd9680a1480 2023-12-11T19:27:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:49 smithi070 ceph-mon[104044]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:27:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:27:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:50.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:49 smithi167 ceph-mon[102750]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:50.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:27:50.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:27:50.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:49 smithi066 ceph-mon[97920]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:27:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:27:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:27:51.160 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:50 smithi167 ceph-mon[102750]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:27:51.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:50 smithi066 ceph-mon[97920]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:27:51.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:50 smithi070 ceph-mon[104044]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:27:52.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:51 smithi167 ceph-mon[102750]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:52.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:51 smithi066 ceph-mon[97920]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:52.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:51 smithi070 ceph-mon[104044]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/1774751859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]: dispatch 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]: dispatch 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]': finished 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: osdmap e36: 6 total, 5 up, 6 in 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:27:54.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:53 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/1629694686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/1774751859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]: dispatch 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]: dispatch 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]': finished 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: osdmap e36: 6 total, 5 up, 6 in 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:27:54.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:53 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/1629694686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:54.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/1774751859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]: dispatch 2023-12-11T19:27:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]: dispatch 2023-12-11T19:27:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3fbfc0-cd26-4ac1-906c-28bd60f02c38"}]': finished 2023-12-11T19:27:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: osdmap e36: 6 total, 5 up, 6 in 2023-12-11T19:27:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:27:54.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:53 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/1629694686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:27:56.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:55 smithi167 ceph-mon[102750]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:56.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:55 smithi066 ceph-mon[97920]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:55 smithi070 ceph-mon[104044]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:58.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:57 smithi167 ceph-mon[102750]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:58.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:57 smithi066 ceph-mon[97920]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:27:58.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:57 smithi070 ceph-mon[104044]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:27:59 smithi167 ceph-mon[102750]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:00.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:27:59 smithi066 ceph-mon[97920]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:00.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:27:59 smithi070 ceph-mon[104044]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:01.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-11T19:28:01.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:01.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-11T19:28:01.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:01.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-11T19:28:01.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:02.266 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:01 smithi167 ceph-mon[102750]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:02.266 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:01 smithi167 ceph-mon[102750]: Deploying daemon osd.5 on smithi167 2023-12-11T19:28:02.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:01 smithi066 ceph-mon[97920]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:02.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:01 smithi066 ceph-mon[97920]: Deploying daemon osd.5 on smithi167 2023-12-11T19:28:02.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:01 smithi070 ceph-mon[104044]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:02.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:01 smithi070 ceph-mon[104044]: Deploying daemon osd.5 on smithi167 2023-12-11T19:28:03.916 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:03 smithi167 ceph-mon[102750]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:04.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:03 smithi066 ceph-mon[97920]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:04.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:03 smithi070 ceph-mon[104044]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:06.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:05 smithi167 ceph-mon[102750]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:05 smithi066 ceph-mon[97920]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:06.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:05 smithi070 ceph-mon[104044]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:07.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:07 smithi167 ceph-mon[102750]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:07.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:07.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:07.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:07.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:07 smithi066 ceph-mon[97920]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:07.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:07.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:07.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:07 smithi070 ceph-mon[104044]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:07.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:07.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:07.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:09 smithi167 ceph-mon[102750]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:09 smithi066 ceph-mon[97920]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:09.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:09 smithi070 ceph-mon[104044]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:09.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:09.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:09.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:09.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:10.106 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2023-12-11T19:28:10.107 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.104+0000 7fd9848ae700 1 -- 172.21.15.167:0/3031409793 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd980109780 con 0x7fd9680a1480 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd9680a1480 msgr2=0x7fd9680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd9680a1480 0x7fd9680a3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd970005950 tx=0x7fd9700058e0 comp rx=0 tx=0).stop 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 msgr2=0x7fd9801a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 0x7fd9801a0ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd978007f10 tx=0x7fd97800d3b0 comp rx=0 tx=0).stop 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 shutdown_connections 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd9680a1480 0x7fd9680a3940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9801031c0 0x7fd9801a0ca0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd980101620 0x7fd980199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 --2- 172.21.15.167:0/3031409793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd980102820 0x7fd980199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 >> 172.21.15.167:0/3031409793 conn(0x7fd9800fcbe0 msgr2=0x7fd980108060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:28:10.109 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 shutdown_connections 2023-12-11T19:28:10.110 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:28:10.107+0000 7fd9858b0700 1 -- 172.21.15.167:0/3031409793 wait complete. 2023-12-11T19:28:10.610 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.5.service 2023-12-11T19:28:10.615 INFO:tasks.cephadm:Deploying osd.6 on smithi066 with /dev/vg_nvme/lv_2... 2023-12-11T19:28:10.615 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-11T19:28:10.690 INFO:journalctl@ceph.osd.5.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:28:11.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:11.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:11.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:11.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:11 smithi167 ceph-mon[102750]: from='osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T19:28:11.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:11 smithi167 ceph-mon[102750]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T19:28:11.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:11.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:11.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:11.363 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:11 smithi070 ceph-mon[104044]: from='osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T19:28:11.363 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:11 smithi070 ceph-mon[104044]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T19:28:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:28:11.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:11.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:11.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:11.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:11 smithi066 ceph-mon[97920]: from='osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T19:28:11.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:11 smithi066 ceph-mon[97920]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T19:28:12.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:12 smithi070 ceph-mon[104044]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:12.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:12 smithi070 ceph-mon[104044]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T19:28:12.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:12 smithi070 ceph-mon[104044]: from='osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:28:12.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:12 smithi070 ceph-mon[104044]: osdmap e37: 6 total, 5 up, 6 in 2023-12-11T19:28:12.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:12 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:12.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:12 smithi070 ceph-mon[104044]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:28:12.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:12 smithi066 ceph-mon[97920]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:12.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:12 smithi066 ceph-mon[97920]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T19:28:12.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:12 smithi066 ceph-mon[97920]: from='osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:28:12.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:12 smithi066 ceph-mon[97920]: osdmap e37: 6 total, 5 up, 6 in 2023-12-11T19:28:12.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:12.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:12 smithi066 ceph-mon[97920]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:28:12.521 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:28:12.103+0000 7f32b0eb3700 -1 osd.5 0 waiting for initial osdmap 2023-12-11T19:28:12.521 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:28:12.122+0000 7f32aa4a8700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:28:12.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-mon[102750]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:12.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-mon[102750]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T19:28:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-mon[102750]: from='osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:28:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-mon[102750]: osdmap e37: 6 total, 5 up, 6 in 2023-12-11T19:28:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:12 smithi167 ceph-mon[102750]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:28:13.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:13 smithi070 ceph-mon[104044]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:28:13.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:13 smithi070 ceph-mon[104044]: osdmap e38: 6 total, 5 up, 6 in 2023-12-11T19:28:13.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:13 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:13.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:13 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:13.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:13 smithi167 ceph-mon[102750]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:28:13.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:13 smithi167 ceph-mon[102750]: osdmap e38: 6 total, 5 up, 6 in 2023-12-11T19:28:13.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:13 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:13.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:13 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-11T19:28:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-11T19:28:13.526 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-12-11T19:28:13.526 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-12-11T19:28:13.526 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0309029 s, 339 MB/s 2023-12-11T19:28:13.526 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-12-11T19:28:13.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:13 smithi066 ceph-mon[97920]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:28:13.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:13 smithi066 ceph-mon[97920]: osdmap e38: 6 total, 5 up, 6 in 2023-12-11T19:28:13.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:13 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:13.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:13 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2023-12-11T19:28:14.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:28:14.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:28:14.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:14.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] boot 2023-12-11T19:28:14.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: osdmap e39: 6 total, 6 up, 6 in 2023-12-11T19:28:14.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:28:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:28:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] boot 2023-12-11T19:28:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: osdmap e39: 6 total, 6 up, 6 in 2023-12-11T19:28:14.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:28:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:28:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-12-11T19:28:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: osd.5 [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] boot 2023-12-11T19:28:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: osdmap e39: 6 total, 6 up, 6 in 2023-12-11T19:28:14.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T19:28:14.829 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: osdmap e40: 6 total, 6 up, 6 in 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi167 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi167 to 8635M 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.354 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:15.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:15.355 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:15 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: osdmap e40: 6 total, 6 up, 6 in 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi167 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:15.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi167 to 8635M 2023-12-11T19:28:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:15 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: osdmap e40: 6 total, 6 up, 6 in 2023-12-11T19:28:15.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi167 2023-12-11T19:28:15.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:15.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:15.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi167 to 8635M 2023-12-11T19:28:15.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:15.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:15.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:15.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:15 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 -- 172.21.15.66:0/2752554872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d0101620 msgr2=0x7fe5d0101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 --2- 172.21.15.66:0/2752554872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d0101620 0x7fe5d0101a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00099c0 tx=0x7fe5c0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:28:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 -- 172.21.15.66:0/2752554872 shutdown_connections 2023-12-11T19:28:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 --2- 172.21.15.66:0/2752554872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d01031c0 0x7fe5d0109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 --2- 172.21.15.66:0/2752554872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d0101620 0x7fe5d0101a40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 --2- 172.21.15.66:0/2752554872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d0102820 0x7fe5d0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 -- 172.21.15.66:0/2752554872 >> 172.21.15.66:0/2752554872 conn(0x7fe5d00fcbe0 msgr2=0x7fe5d00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:28:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 -- 172.21.15.66:0/2752554872 shutdown_connections 2023-12-11T19:28:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.241+0000 7fe5d482e700 1 -- 172.21.15.66:0/2752554872 wait complete. 2023-12-11T19:28:16.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 Processor -- start 2023-12-11T19:28:16.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 -- start start 2023-12-11T19:28:16.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 0x7fe5d0075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:16.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d0102820 0x7fe5d00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:16.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d01031c0 0x7fe5d0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00746a0 con 0x7fe5d01031c0 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0074810 con 0x7fe5d0101620 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.242+0000 7fe5d482e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d01a14b0 con 0x7fe5d0102820 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ced9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d01031c0 0x7fe5d0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 0x7fe5d0075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 0x7fe5d0075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:48054/0 (socket says 172.21.15.66:48054) 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ce59c700 1 -- 172.21.15.66:0/3127962977 learned_addr learned my addr 172.21.15.66:0/3127962977 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:28:16.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5cdd9b700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d0102820 0x7fe5d00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ced9d700 1 -- 172.21.15.66:0/3127962977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d0102820 msgr2=0x7fe5d00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ced9d700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d0102820 0x7fe5d00739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ced9d700 1 -- 172.21.15.66:0/3127962977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 msgr2=0x7fe5d0075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ced9d700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 0x7fe5d0075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ced9d700 1 -- 172.21.15.66:0/3127962977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00095f0 con 0x7fe5d01031c0 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.243+0000 7fe5ce59c700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 0x7fe5d0075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:28:16.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.244+0000 7fe5ced9d700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d01031c0 0x7fe5d0073f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800cc50 tx=0x7fe5c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:28:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.244+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800aa90 con 0x7fe5d01031c0 2023-12-11T19:28:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.244+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5c800abf0 con 0x7fe5d01031c0 2023-12-11T19:28:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.244+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80115e0 con 0x7fe5d01031c0 2023-12-11T19:28:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.244+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d01a1740 con 0x7fe5d01031c0 2023-12-11T19:28:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.244+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d01a1bf0 con 0x7fe5d01031c0 2023-12-11T19:28:16.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.245+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d0108b00 con 0x7fe5d01031c0 2023-12-11T19:28:16.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.247+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe5c8012420 con 0x7fe5d01031c0 2023-12-11T19:28:16.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.249+0000 7fe5bf7fe700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5b80a1480 0x7fe5b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:16.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.249+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fe5c80cc500 con 0x7fe5d01031c0 2023-12-11T19:28:16.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.249+0000 7fe5ce59c700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5b80a1480 0x7fe5b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:16.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.249+0000 7fe5ce59c700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5b80a1480 0x7fe5b80a3940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0006010 tx=0x7fe5c001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:28:16.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.252+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe5c8096980 con 0x7fe5d01031c0 2023-12-11T19:28:16.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:16 smithi066 ceph-mon[97920]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:16.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:16 smithi066 ceph-mon[97920]: osdmap e41: 6 total, 6 up, 6 in 2023-12-11T19:28:16.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:16.473+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5d0061a10 con 0x7fe5b80a1480 2023-12-11T19:28:16.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:16 smithi167 ceph-mon[102750]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:16.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:16 smithi167 ceph-mon[102750]: osdmap e41: 6 total, 6 up, 6 in 2023-12-11T19:28:16.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:16 smithi070 ceph-mon[104044]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:16.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:16 smithi070 ceph-mon[104044]: osdmap e41: 6 total, 6 up, 6 in 2023-12-11T19:28:17.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:17 smithi167 ceph-mon[102750]: osdmap e42: 6 total, 6 up, 6 in 2023-12-11T19:28:17.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:17 smithi167 ceph-mon[102750]: pgmap v155: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:28:17.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:17 smithi167 ceph-mon[102750]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:28:17.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:28:17.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:28:17.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:17.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:17 smithi066 ceph-mon[97920]: osdmap e42: 6 total, 6 up, 6 in 2023-12-11T19:28:17.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:17 smithi066 ceph-mon[97920]: pgmap v155: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:28:17.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:17 smithi066 ceph-mon[97920]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:28:17.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:28:17.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:28:17.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:17.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:17 smithi070 ceph-mon[104044]: osdmap e42: 6 total, 6 up, 6 in 2023-12-11T19:28:17.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:17 smithi070 ceph-mon[104044]: pgmap v155: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:28:17.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:17 smithi070 ceph-mon[104044]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:28:17.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:28:17.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:28:17.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:18.266 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:18 smithi066 ceph-mon[97920]: osdmap e43: 6 total, 6 up, 6 in 2023-12-11T19:28:18.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:18 smithi167 ceph-mon[102750]: osdmap e43: 6 total, 6 up, 6 in 2023-12-11T19:28:18.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:18 smithi070 ceph-mon[104044]: osdmap e43: 6 total, 6 up, 6 in 2023-12-11T19:28:19.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:19 smithi167 ceph-mon[102750]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 140 KiB/s, 0 objects/s recovering 2023-12-11T19:28:19.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:19 smithi066 ceph-mon[97920]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 140 KiB/s, 0 objects/s recovering 2023-12-11T19:28:19.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:19 smithi070 ceph-mon[104044]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 140 KiB/s, 0 objects/s recovering 2023-12-11T19:28:20.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:20 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1940595964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb558c6d-a82c-4255-a040-4432a98c2aa1"}]: dispatch 2023-12-11T19:28:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:20 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1940595964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb558c6d-a82c-4255-a040-4432a98c2aa1"}]': finished 2023-12-11T19:28:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:20 smithi167 ceph-mon[102750]: osdmap e44: 7 total, 6 up, 7 in 2023-12-11T19:28:20.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:20 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:20.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:20 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1940595964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb558c6d-a82c-4255-a040-4432a98c2aa1"}]: dispatch 2023-12-11T19:28:20.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:20 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1940595964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb558c6d-a82c-4255-a040-4432a98c2aa1"}]': finished 2023-12-11T19:28:20.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:20 smithi066 ceph-mon[97920]: osdmap e44: 7 total, 6 up, 7 in 2023-12-11T19:28:20.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:20 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:20.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:20 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1940595964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb558c6d-a82c-4255-a040-4432a98c2aa1"}]: dispatch 2023-12-11T19:28:20.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:20 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1940595964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb558c6d-a82c-4255-a040-4432a98c2aa1"}]': finished 2023-12-11T19:28:20.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:20 smithi070 ceph-mon[104044]: osdmap e44: 7 total, 6 up, 7 in 2023-12-11T19:28:20.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:20 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:21 smithi167 ceph-mon[102750]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-12-11T19:28:21.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:21 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3480141184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:28:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:21 smithi066 ceph-mon[97920]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-12-11T19:28:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:21 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3480141184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:28:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:21 smithi070 ceph-mon[104044]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-12-11T19:28:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:21 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3480141184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:28:23.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:23 smithi066 ceph-mon[97920]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:23.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:23 smithi167 ceph-mon[102750]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:23.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:23 smithi070 ceph-mon[104044]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:25.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:25 smithi066 ceph-mon[97920]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:25.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:25 smithi167 ceph-mon[102750]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:25.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:25 smithi070 ceph-mon[104044]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:27 smithi167 ceph-mon[102750]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-11T19:28:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:27 smithi066 ceph-mon[97920]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-11T19:28:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:27.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:27 smithi070 ceph-mon[104044]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:27.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-11T19:28:27.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:28.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:28 smithi066 ceph-mon[97920]: Deploying daemon osd.6 on smithi066 2023-12-11T19:28:28.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:28 smithi167 ceph-mon[102750]: Deploying daemon osd.6 on smithi066 2023-12-11T19:28:28.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:28 smithi070 ceph-mon[104044]: Deploying daemon osd.6 on smithi066 2023-12-11T19:28:29.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:29 smithi167 ceph-mon[102750]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:29.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:29 smithi066 ceph-mon[97920]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:29 smithi070 ceph-mon[104044]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:31.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:31 smithi167 ceph-mon[102750]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:31.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:31 smithi066 ceph-mon[97920]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:31.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:31 smithi070 ceph-mon[104044]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:33.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:33 smithi066 ceph-mon[97920]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:33.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:33 smithi167 ceph-mon[102750]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:33.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:33 smithi070 ceph-mon[104044]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:35.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:35.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:35.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:35.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:35.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:35.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:35.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:35.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:35.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:35.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:35 smithi066 ceph-mon[97920]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:36.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:35 smithi070 ceph-mon[104044]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:36.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:35 smithi167 ceph-mon[102750]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:37.336 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 6 on host 'smithi066' 2023-12-11T19:28:37.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.334+0000 7fe5bf7fe700 1 -- 172.21.15.66:0/3127962977 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe5d0061a10 con 0x7fe5b80a1480 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5b80a1480 msgr2=0x7fe5b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5b80a1480 0x7fe5b80a3940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0006010 tx=0x7fe5c001a040 comp rx=0 tx=0).stop 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d01031c0 msgr2=0x7fe5d0073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d01031c0 0x7fe5d0073f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800cc50 tx=0x7fe5c800cf60 comp rx=0 tx=0).stop 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 shutdown_connections 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5b80a1480 0x7fe5b80a3940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101620 0x7fe5d0075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5d01031c0 0x7fe5d0073f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 --2- 172.21.15.66:0/3127962977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d0102820 0x7fe5d00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:37.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 >> 172.21.15.66:0/3127962977 conn(0x7fe5d00fcbe0 msgr2=0x7fe5d0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:28:37.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 shutdown_connections 2023-12-11T19:28:37.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:28:37.337+0000 7fe5d482e700 1 -- 172.21.15.66:0/3127962977 wait complete. 2023-12-11T19:28:37.786 DEBUG:teuthology.orchestra.run.smithi066:osd.6> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.6.service 2023-12-11T19:28:37.791 INFO:tasks.cephadm:Deploying osd.7 on smithi070 with /dev/vg_nvme/lv_2... 2023-12-11T19:28:37.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-11T19:28:37.865 INFO:journalctl@ceph.osd.6.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.065 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:38.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:37 smithi066 ceph-mon[97920]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-11T19:28:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:38.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:38.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:37 smithi070 ceph-mon[104044]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:28:38.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:37 smithi167 ceph-mon[102750]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-11T19:28:38.574 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:28:39.329 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:28:39.312+0000 7f0cd4f91700 -1 osd.6 0 waiting for initial osdmap 2023-12-11T19:28:39.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-mon[97920]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:39.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-mon[97920]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-11T19:28:39.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-mon[97920]: osdmap e45: 7 total, 6 up, 7 in 2023-12-11T19:28:39.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-mon[97920]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:28:39.329 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:39.475 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:39 smithi070 ceph-mon[104044]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:39.475 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:39 smithi070 ceph-mon[104044]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-11T19:28:39.476 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:39 smithi070 ceph-mon[104044]: osdmap e45: 7 total, 6 up, 7 in 2023-12-11T19:28:39.476 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:39 smithi070 ceph-mon[104044]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:28:39.476 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:39 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:39.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:39 smithi167 ceph-mon[102750]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:39.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:39 smithi167 ceph-mon[102750]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-11T19:28:39.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:39 smithi167 ceph-mon[102750]: osdmap e45: 7 total, 6 up, 7 in 2023-12-11T19:28:39.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:39 smithi167 ceph-mon[102750]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:28:39.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:39 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:39.813 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:28:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:28:39.326+0000 7f0cce586700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:28:40.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:40 smithi066 ceph-mon[97920]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:28:40.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:40 smithi066 ceph-mon[97920]: osdmap e46: 7 total, 6 up, 7 in 2023-12-11T19:28:40.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:40.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:40.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:40 smithi070 ceph-mon[104044]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:28:40.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:40 smithi070 ceph-mon[104044]: osdmap e46: 7 total, 6 up, 7 in 2023-12-11T19:28:40.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:40.612 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-11T19:28:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-11T19:28:40.693 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-12-11T19:28:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-12-11T19:28:40.693 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0319549 s, 328 MB/s 2023-12-11T19:28:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-12-11T19:28:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:40 smithi167 ceph-mon[102750]: from='osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:28:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:40 smithi167 ceph-mon[102750]: osdmap e46: 7 total, 6 up, 7 in 2023-12-11T19:28:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:41.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2023-12-11T19:28:41.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] boot 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: osdmap e47: 7 total, 7 up, 7 in 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:41.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:41.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:28:41.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] boot 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: osdmap e47: 7 total, 7 up, 7 in 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:41.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: osd.6 [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] boot 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: osdmap e47: 7 total, 7 up, 7 in 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:28:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:28:42.038 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:28:42.592 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:42 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi066 2023-12-11T19:28:42.592 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:42 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi066 to 5757M 2023-12-11T19:28:42.592 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:42 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:42.592 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:42 smithi070 ceph-mon[104044]: osdmap e48: 7 total, 7 up, 7 in 2023-12-11T19:28:42.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:42 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi066 2023-12-11T19:28:42.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:42 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi066 to 5757M 2023-12-11T19:28:42.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:42 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:42.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:42 smithi167 ceph-mon[102750]: osdmap e48: 7 total, 7 up, 7 in 2023-12-11T19:28:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:42 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi066 2023-12-11T19:28:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:42 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi066 to 5757M 2023-12-11T19:28:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:42 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:28:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:42 smithi066 ceph-mon[97920]: osdmap e48: 7 total, 7 up, 7 in 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 -- 172.21.15.70:0/4099965855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 msgr2=0x7fe298109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/4099965855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298109a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe29000b0d0 tx=0x7fe29000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 -- 172.21.15.70:0/4099965855 shutdown_connections 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/4099965855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298109a70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/4099965855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe298101620 0x7fe298101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/4099965855 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe298102820 0x7fe298102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 -- 172.21.15.70:0/4099965855 >> 172.21.15.70:0/4099965855 conn(0x7fe2980fcbe0 msgr2=0x7fe2980ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 -- 172.21.15.70:0/4099965855 shutdown_connections 2023-12-11T19:28:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.504+0000 7fe29eb4c700 1 -- 172.21.15.70:0/4099965855 wait complete. 2023-12-11T19:28:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 Processor -- start 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 -- start start 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe298101620 0x7fe298075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe298102820 0x7fe2980739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980746a0 con 0x7fe298102820 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298074810 con 0x7fe2981031c0 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.505+0000 7fe29eb4c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2981a1550 con 0x7fe298101620 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49194/0 (socket says 172.21.15.70:49194) 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 -- 172.21.15.70:0/1789903139 learned_addr learned my addr 172.21.15.70:0/1789903139 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29c8e8700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe298101620 0x7fe298075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe297fff700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe298102820 0x7fe2980739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 -- 172.21.15.70:0/1789903139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe298101620 msgr2=0x7fe298075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe298101620 0x7fe298075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 -- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe298102820 msgr2=0x7fe2980739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:28:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe298102820 0x7fe2980739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:28:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 -- 172.21.15.70:0/1789903139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe290009d00 con 0x7fe2981031c0 2023-12-11T19:28:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe29d0e9700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298073f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2900079c0 tx=0x7fe29000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:28:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.506+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29000e040 con 0x7fe2981031c0 2023-12-11T19:28:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.507+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe290004e60 con 0x7fe2981031c0 2023-12-11T19:28:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.507+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29001b6b0 con 0x7fe2981031c0 2023-12-11T19:28:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.507+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2981a1780 con 0x7fe2981031c0 2023-12-11T19:28:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.507+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2981a1bf0 con 0x7fe2981031c0 2023-12-11T19:28:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.508+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe298108ad0 con 0x7fe2981031c0 2023-12-11T19:28:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.509+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe29001da10 con 0x7fe2981031c0 2023-12-11T19:28:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.511+0000 7fe295ffb700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2880a1500 0x7fe2880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:28:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.511+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fe2900cda40 con 0x7fe2981031c0 2023-12-11T19:28:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.511+0000 7fe29c8e8700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2880a1500 0x7fe2880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:28:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.512+0000 7fe29c8e8700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2880a1500 0x7fe2880a39c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005950 tx=0x7fe28c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:28:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.515+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe290097da0 con 0x7fe2981031c0 2023-12-11T19:28:43.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:43 smithi070 ceph-mon[104044]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:43.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:28:43.723+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe29802d3b0 con 0x7fe2880a1500 2023-12-11T19:28:43.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:43 smithi167 ceph-mon[102750]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:43.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:43 smithi066 ceph-mon[97920]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:44.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:44 smithi167 ceph-mon[102750]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:28:44.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:28:44.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:28:44.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:44.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:44 smithi066 ceph-mon[97920]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:28:44.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:28:44.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:28:44.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:44.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:44 smithi070 ceph-mon[104044]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:28:44.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:28:44.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:28:44.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:45.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:45 smithi167 ceph-mon[102750]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:45.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:45 smithi066 ceph-mon[97920]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:45.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:45 smithi070 ceph-mon[104044]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:47.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:47 smithi167 ceph-mon[102750]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:47.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:47 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/120884225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]: dispatch 2023-12-11T19:28:47.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:47 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]: dispatch 2023-12-11T19:28:47.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:47 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]': finished 2023-12-11T19:28:47.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:47 smithi167 ceph-mon[102750]: osdmap e49: 8 total, 7 up, 8 in 2023-12-11T19:28:47.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:47 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:28:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:47 smithi066 ceph-mon[97920]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:47 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/120884225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]: dispatch 2023-12-11T19:28:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:47 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]: dispatch 2023-12-11T19:28:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:47 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]': finished 2023-12-11T19:28:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:47 smithi066 ceph-mon[97920]: osdmap e49: 8 total, 7 up, 8 in 2023-12-11T19:28:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:47 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:28:47.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:47 smithi070 ceph-mon[104044]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:47.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:47 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/120884225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]: dispatch 2023-12-11T19:28:47.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:47 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]: dispatch 2023-12-11T19:28:47.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:47 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f"}]': finished 2023-12-11T19:28:47.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:47 smithi070 ceph-mon[104044]: osdmap e49: 8 total, 7 up, 8 in 2023-12-11T19:28:47.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:47 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:28:48.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:48 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/3890415246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:28:48.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:48 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/3890415246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:28:48.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:48 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/3890415246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:28:49.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:49 smithi167 ceph-mon[102750]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:49.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:49 smithi066 ceph-mon[97920]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:49.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:49 smithi070 ceph-mon[104044]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:51.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:51 smithi167 ceph-mon[102750]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:51 smithi066 ceph-mon[97920]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:51 smithi070 ceph-mon[104044]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:53.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:53 smithi167 ceph-mon[102750]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:53 smithi066 ceph-mon[97920]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:53 smithi070 ceph-mon[104044]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:55.755 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:55 smithi070 ceph-mon[104044]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:55.755 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-11T19:28:55.755 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:55.755 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:55 smithi070 ceph-mon[104044]: Deploying daemon osd.7 on smithi070 2023-12-11T19:28:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:55 smithi167 ceph-mon[102750]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-11T19:28:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:55 smithi167 ceph-mon[102750]: Deploying daemon osd.7 on smithi070 2023-12-11T19:28:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:55 smithi066 ceph-mon[97920]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-11T19:28:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:28:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:55 smithi066 ceph-mon[97920]: Deploying daemon osd.7 on smithi070 2023-12-11T19:28:57.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:57 smithi167 ceph-mon[102750]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:57.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:57 smithi066 ceph-mon[97920]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:57.848 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:57 smithi070 ceph-mon[104044]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:28:59 smithi167 ceph-mon[102750]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:59.794 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:28:59 smithi070 ceph-mon[104044]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:28:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:28:59 smithi066 ceph-mon[97920]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:02.328 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:02 smithi070 ceph-mon[104044]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:02.328 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:02 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:02.328 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:02 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:02.328 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:02 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:02.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:02 smithi167 ceph-mon[102750]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:02.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:02.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:02.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:02.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:02 smithi066 ceph-mon[97920]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:02.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:02.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:02.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:04.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:04 smithi070 ceph-mon[104044]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:04.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:04 smithi167 ceph-mon[102750]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:04.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:04 smithi066 ceph-mon[97920]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.608+0000 7fe295ffb700 1 -- 172.21.15.70:0/1789903139 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe29802d3b0 con 0x7fe2880a1500 2023-12-11T19:29:04.609 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 7 on host 'smithi070' 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2880a1500 msgr2=0x7fe2880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2880a1500 0x7fe2880a39c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005950 tx=0x7fe28c0058e0 comp rx=0 tx=0).stop 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 msgr2=0x7fe298073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298073f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2900079c0 tx=0x7fe29000cd70 comp rx=0 tx=0).stop 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 shutdown_connections 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2880a1500 0x7fe2880a39c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2981031c0 0x7fe298073f00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe298101620 0x7fe298075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 --2- 172.21.15.70:0/1789903139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe298102820 0x7fe2980739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 >> 172.21.15.70:0/1789903139 conn(0x7fe2980fcbe0 msgr2=0x7fe298105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 shutdown_connections 2023-12-11T19:29:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:29:04.611+0000 7fe29eb4c700 1 -- 172.21.15.70:0/1789903139 wait complete. 2023-12-11T19:29:05.078 DEBUG:teuthology.orchestra.run.smithi070:osd.7> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.7.service 2023-12-11T19:29:05.084 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2023-12-11T19:29:05.085 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-11T19:29:05.168 INFO:journalctl@ceph.osd.7.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T19:29:05.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:05 smithi070 ceph-mon[104044]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T19:29:05.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T19:29:05.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:05 smithi167 ceph-mon[102750]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:05.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T19:29:05.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:05 smithi066 ceph-mon[97920]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T19:29:05.877 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:29:06.348 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-mon[104044]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:06.349 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-mon[104044]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-11T19:29:06.349 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-mon[104044]: osdmap e50: 8 total, 7 up, 8 in 2023-12-11T19:29:06.349 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-mon[104044]: from='osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:29:06.349 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:06.349 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-mon[104044]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:29:06.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:06 smithi167 ceph-mon[102750]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:06.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:06 smithi167 ceph-mon[102750]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-11T19:29:06.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:06 smithi167 ceph-mon[102750]: osdmap e50: 8 total, 7 up, 8 in 2023-12-11T19:29:06.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:06 smithi167 ceph-mon[102750]: from='osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:29:06.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:06.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:06 smithi167 ceph-mon[102750]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:29:06.565 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:06 smithi066 ceph-mon[97920]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:06.565 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:06 smithi066 ceph-mon[97920]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-11T19:29:06.565 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:06 smithi066 ceph-mon[97920]: osdmap e50: 8 total, 7 up, 8 in 2023-12-11T19:29:06.565 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:06 smithi066 ceph-mon[97920]: from='osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:29:06.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:06.566 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:06 smithi066 ceph-mon[97920]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:29:07.110 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:29:06.628+0000 7f8f2b33a700 -1 osd.7 0 waiting for initial osdmap 2023-12-11T19:29:07.110 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:29:06 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:29:06.640+0000 7f8f25130700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:29:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:07 smithi167 ceph-mon[102750]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:07 smithi167 ceph-mon[102750]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:29:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:07 smithi167 ceph-mon[102750]: osdmap e51: 8 total, 7 up, 8 in 2023-12-11T19:29:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:08.022 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-11T19:29:08.022 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-11T19:29:08.022 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-12-11T19:29:08.022 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-12-11T19:29:08.023 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0457556 s, 229 MB/s 2023-12-11T19:29:08.023 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-12-11T19:29:08.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:07 smithi066 ceph-mon[97920]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:08.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:07 smithi066 ceph-mon[97920]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:29:08.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:07 smithi066 ceph-mon[97920]: osdmap e51: 8 total, 7 up, 8 in 2023-12-11T19:29:08.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:08.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:08.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:07 smithi070 ceph-mon[104044]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:08.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:07 smithi070 ceph-mon[104044]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:29:08.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:07 smithi070 ceph-mon[104044]: osdmap e51: 8 total, 7 up, 8 in 2023-12-11T19:29:08.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:08.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:08.572 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2023-12-11T19:29:08.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:08 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:29:08.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:08 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:29:08.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:08 smithi167 ceph-mon[102750]: osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] boot 2023-12-11T19:29:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:08 smithi167 ceph-mon[102750]: osdmap e52: 8 total, 8 up, 8 in 2023-12-11T19:29:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:08 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:09.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:08 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:29:09.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:08 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:29:09.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:08 smithi066 ceph-mon[97920]: osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] boot 2023-12-11T19:29:09.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:08 smithi066 ceph-mon[97920]: osdmap e52: 8 total, 8 up, 8 in 2023-12-11T19:29:09.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:08 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:09.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:08 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:29:09.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:08 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:29:09.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:08 smithi070 ceph-mon[104044]: osd.7 [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] boot 2023-12-11T19:29:09.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:08 smithi070 ceph-mon[104044]: osdmap e52: 8 total, 8 up, 8 in 2023-12-11T19:29:09.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:08 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T19:29:09.358 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: osdmap e53: 8 total, 8 up, 8 in 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi070 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:09.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:09.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:09.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi070 to 5757M 2023-12-11T19:29:09.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:09.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:09.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:09.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:10.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: osdmap e53: 8 total, 8 up, 8 in 2023-12-11T19:29:10.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi070 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi070 to 5757M 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:10.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T19:29:10.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: osdmap e53: 8 total, 8 up, 8 in 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi070 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi070 to 5757M 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:10.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:10.846 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 -- 172.21.15.167:0/859018944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81031c0 msgr2=0x7f45b8109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:10.846 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 --2- 172.21.15.167:0/859018944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81031c0 0x7f45b8109a70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f45b000b0a0 tx=0x7f45b000b3b0 comp rx=0 tx=0).stop 2023-12-11T19:29:10.846 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 -- 172.21.15.167:0/859018944 shutdown_connections 2023-12-11T19:29:10.847 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 --2- 172.21.15.167:0/859018944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45b8101620 0x7f45b8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:10.847 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 --2- 172.21.15.167:0/859018944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b81031c0 0x7f45b8109a70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:10.847 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 --2- 172.21.15.167:0/859018944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b8102820 0x7f45b8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:10.847 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 -- 172.21.15.167:0/859018944 >> 172.21.15.167:0/859018944 conn(0x7f45b80fcbe0 msgr2=0x7f45b80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:29:10.847 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 -- 172.21.15.167:0/859018944 shutdown_connections 2023-12-11T19:29:10.847 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.844+0000 7f45c0522700 1 -- 172.21.15.167:0/859018944 wait complete. 2023-12-11T19:29:10.848 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.845+0000 7f45c0522700 1 Processor -- start 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.845+0000 7f45c0522700 1 -- start start 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45c0522700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 0x7f45b8197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45c0522700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45b8102820 0x7f45b8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45c0522700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b81031c0 0x7f45b819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45c0522700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b819f0d0 con 0x7f45b81031c0 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45c0522700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b819f210 con 0x7f45b8102820 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45c0522700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b819f3a0 con 0x7f45b8101620 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45be2be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 0x7f45b8197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45be2be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 0x7f45b8197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34228/0 (socket says 172.21.15.167:34228) 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45be2be700 1 -- 172.21.15.167:0/2522714742 learned_addr learned my addr 172.21.15.167:0/2522714742 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45beabf700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b81031c0 0x7f45b819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:10.849 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.846+0000 7f45bdabd700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45b8102820 0x7f45b8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45be2be700 1 -- 172.21.15.167:0/2522714742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45b8102820 msgr2=0x7f45b8197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45be2be700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45b8102820 0x7f45b8197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45be2be700 1 -- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b81031c0 msgr2=0x7f45b819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45be2be700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b81031c0 0x7f45b819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45be2be700 1 -- 172.21.15.167:0/2522714742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0009d00 con 0x7f45b8101620 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45be2be700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 0x7f45b8197590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45b400d9b0 tx=0x7f45b400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400be80 con 0x7f45b8101620 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45b40044b0 con 0x7f45b8101620 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b8073860 con 0x7f45b8101620 2023-12-11T19:29:10.850 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4003aa0 con 0x7f45b8101620 2023-12-11T19:29:10.851 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.847+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b8073db0 con 0x7f45b8101620 2023-12-11T19:29:10.851 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.848+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b8108aa0 con 0x7f45b8101620 2023-12-11T19:29:10.853 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.852+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f45b4009940 con 0x7f45b8101620 2023-12-11T19:29:10.855 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.853+0000 7f45af7fe700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45a40a1500 0x7f45a40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:10.855 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.853+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f45b40cc230 con 0x7f45b8101620 2023-12-11T19:29:10.855 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.854+0000 7f45bdabd700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45a40a1500 0x7f45a40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:10.856 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.855+0000 7f45bdabd700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45a40a1500 0x7f45a40a39c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005fd0 tx=0x7f45a8005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:29:10.857 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:10.856+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f45b40963d0 con 0x7f45b8101620 2023-12-11T19:29:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:10 smithi167 ceph-mon[102750]: osdmap e54: 8 total, 8 up, 8 in 2023-12-11T19:29:11.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:10 smithi066 ceph-mon[97920]: osdmap e54: 8 total, 8 up, 8 in 2023-12-11T19:29:11.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:11.065+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f45b802d380 con 0x7f45a40a1500 2023-12-11T19:29:11.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:10 smithi070 ceph-mon[104044]: osdmap e54: 8 total, 8 up, 8 in 2023-12-11T19:29:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:11 smithi167 ceph-mon[102750]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:11 smithi167 ceph-mon[102750]: osdmap e55: 8 total, 8 up, 8 in 2023-12-11T19:29:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:11 smithi167 ceph-mon[102750]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:29:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:29:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:29:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:11 smithi066 ceph-mon[97920]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:11 smithi066 ceph-mon[97920]: osdmap e55: 8 total, 8 up, 8 in 2023-12-11T19:29:12.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:11 smithi066 ceph-mon[97920]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:29:12.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:29:12.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:29:12.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:12.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:11 smithi070 ceph-mon[104044]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:12.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:11 smithi070 ceph-mon[104044]: osdmap e55: 8 total, 8 up, 8 in 2023-12-11T19:29:12.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:11 smithi070 ceph-mon[104044]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:29:12.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:29:12.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:29:12.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:12.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:12 smithi167 ceph-mon[102750]: osdmap e56: 8 total, 8 up, 8 in 2023-12-11T19:29:13.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:12 smithi066 ceph-mon[97920]: osdmap e56: 8 total, 8 up, 8 in 2023-12-11T19:29:13.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:12 smithi070 ceph-mon[104044]: osdmap e56: 8 total, 8 up, 8 in 2023-12-11T19:29:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:13 smithi167 ceph-mon[102750]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:13 smithi167 ceph-mon[102750]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-12-11T19:29:14.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:13 smithi066 ceph-mon[97920]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:14.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:13 smithi066 ceph-mon[97920]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-12-11T19:29:14.115 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:13 smithi070 ceph-mon[104044]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:14.115 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:13 smithi070 ceph-mon[104044]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-12-11T19:29:15.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:14 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/1158835023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]: dispatch 2023-12-11T19:29:15.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:14 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]: dispatch 2023-12-11T19:29:15.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:14 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]': finished 2023-12-11T19:29:15.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:14 smithi167 ceph-mon[102750]: osdmap e57: 9 total, 8 up, 9 in 2023-12-11T19:29:15.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:15.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:14 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/1158835023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]: dispatch 2023-12-11T19:29:15.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:14 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]: dispatch 2023-12-11T19:29:15.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:14 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]': finished 2023-12-11T19:29:15.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:14 smithi066 ceph-mon[97920]: osdmap e57: 9 total, 8 up, 9 in 2023-12-11T19:29:15.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:15.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:14 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/1158835023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]: dispatch 2023-12-11T19:29:15.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:14 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]: dispatch 2023-12-11T19:29:15.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:14 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d35b137-1e21-4e15-b0b8-4f0c6c693e73"}]': finished 2023-12-11T19:29:15.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:14 smithi070 ceph-mon[104044]: osdmap e57: 9 total, 8 up, 9 in 2023-12-11T19:29:15.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:16.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:15 smithi167 ceph-mon[102750]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:16.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:15 smithi167 ceph-mon[102750]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-12-11T19:29:16.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:15 smithi167 ceph-mon[102750]: Cluster is now healthy 2023-12-11T19:29:16.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:15 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/551511827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:29:16.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:15 smithi066 ceph-mon[97920]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:16.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:15 smithi066 ceph-mon[97920]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-12-11T19:29:16.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:15 smithi066 ceph-mon[97920]: Cluster is now healthy 2023-12-11T19:29:16.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:15 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/551511827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:29:16.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:15 smithi070 ceph-mon[104044]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:16.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:15 smithi070 ceph-mon[104044]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-12-11T19:29:16.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:15 smithi070 ceph-mon[104044]: Cluster is now healthy 2023-12-11T19:29:16.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:15 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/551511827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:29:18.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:17 smithi167 ceph-mon[102750]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:18.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:17 smithi066 ceph-mon[97920]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:18.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:17 smithi070 ceph-mon[104044]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:19.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:19 smithi167 ceph-mon[102750]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:20.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:19 smithi066 ceph-mon[97920]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:20.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:19 smithi070 ceph-mon[104044]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:21 smithi167 ceph-mon[102750]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:22.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:21 smithi066 ceph-mon[97920]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:22.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:21 smithi070 ceph-mon[104044]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:22.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-11T19:29:22.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:22 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:22.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:22 smithi167 ceph-mon[102750]: Deploying daemon osd.8 on smithi167 2023-12-11T19:29:23.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-11T19:29:23.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:22 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:23.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:22 smithi066 ceph-mon[97920]: Deploying daemon osd.8 on smithi167 2023-12-11T19:29:23.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-11T19:29:23.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:22 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:23.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:22 smithi070 ceph-mon[104044]: Deploying daemon osd.8 on smithi167 2023-12-11T19:29:24.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:23 smithi167 ceph-mon[102750]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:24.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:23 smithi066 ceph-mon[97920]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:24.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:23 smithi070 ceph-mon[104044]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:25.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:25 smithi167 ceph-mon[102750]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:26.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:25 smithi066 ceph-mon[97920]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:26.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:25 smithi070 ceph-mon[104044]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:28.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:27 smithi167 ceph-mon[102750]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:27 smithi066 ceph-mon[97920]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:27 smithi070 ceph-mon[104044]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:29.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:29 smithi167 ceph-mon[102750]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:29.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:29 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:29.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:29 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:29.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:29 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:29.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:29 smithi066 ceph-mon[97920]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:29.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:29 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:29.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:29 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:29.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:29 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:29 smithi070 ceph-mon[104044]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:29 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:29 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:29 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:31 smithi167 ceph-mon[102750]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:31.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:31 smithi066 ceph-mon[97920]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:31.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:31.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:31.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:31.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:31.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:31 smithi070 ceph-mon[104044]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:31.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:31.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:31.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:31.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:32.148 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.145+0000 7f45af7fe700 1 -- 172.21.15.167:0/2522714742 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f45b802d380 con 0x7f45a40a1500 2023-12-11T19:29:32.148 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45a40a1500 msgr2=0x7f45a40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45a40a1500 0x7f45a40a39c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005fd0 tx=0x7f45a8005f60 comp rx=0 tx=0).stop 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 msgr2=0x7f45b8197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 0x7f45b8197590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45b400d9b0 tx=0x7f45b400dd70 comp rx=0 tx=0).stop 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 shutdown_connections 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45a40a1500 0x7f45a40a39c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45b81031c0 0x7f45b819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45b8101620 0x7f45b8197590 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 --2- 172.21.15.167:0/2522714742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45b8102820 0x7f45b8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 >> 172.21.15.167:0/2522714742 conn(0x7f45b80fcbe0 msgr2=0x7f45b8105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 shutdown_connections 2023-12-11T19:29:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:29:32.148+0000 7f45c0522700 1 -- 172.21.15.167:0/2522714742 wait complete. 2023-12-11T19:29:32.691 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.8.service 2023-12-11T19:29:32.696 INFO:tasks.cephadm:Deploying osd.9 on smithi066 with /dev/vg_nvme/lv_1... 2023-12-11T19:29:32.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-11T19:29:32.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:32 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:32 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:32 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:32.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:32 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:32 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.767 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:32 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.768 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:32 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:32.768 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:32 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.777 INFO:journalctl@ceph.osd.8.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:29:32.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:32 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:32 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:32.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:32 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:32.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:32 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:29:33.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:33 smithi167 ceph-mon[102750]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:33.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:33 smithi167 ceph-mon[102750]: from='osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T19:29:33.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:33 smithi167 ceph-mon[102750]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T19:29:33.768 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:33 smithi066 ceph-mon[97920]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:33.769 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:33 smithi066 ceph-mon[97920]: from='osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T19:29:33.769 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:33 smithi066 ceph-mon[97920]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T19:29:33.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:33 smithi070 ceph-mon[104044]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:33.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:33 smithi070 ceph-mon[104044]: from='osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T19:29:33.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:33 smithi070 ceph-mon[104044]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T19:29:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:34 smithi066 ceph-mon[97920]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T19:29:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:34 smithi066 ceph-mon[97920]: osdmap e58: 9 total, 8 up, 9 in 2023-12-11T19:29:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:34 smithi066 ceph-mon[97920]: from='osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:29:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:34 smithi066 ceph-mon[97920]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:29:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-mon[102750]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T19:29:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-mon[102750]: osdmap e58: 9 total, 8 up, 9 in 2023-12-11T19:29:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-mon[102750]: from='osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:29:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:34.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-mon[102750]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:29:34.772 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:29:34.540+0000 7ffa1eae6700 -1 osd.8 0 waiting for initial osdmap 2023-12-11T19:29:34.772 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:29:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:29:34.555+0000 7ffa188dc700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:29:34.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:34 smithi070 ceph-mon[104044]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T19:29:34.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:34 smithi070 ceph-mon[104044]: osdmap e58: 9 total, 8 up, 9 in 2023-12-11T19:29:34.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:34 smithi070 ceph-mon[104044]: from='osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:29:34.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:34.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:34 smithi070 ceph-mon[104044]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:29:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-11T19:29:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-11T19:29:35.536 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-12-11T19:29:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-12-11T19:29:35.536 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.05497 s, 191 MB/s 2023-12-11T19:29:35.536 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: osdmap e59: 9 total, 8 up, 9 in 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:35.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:35 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:35.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:29:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:29:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:29:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: osdmap e59: 9 total, 8 up, 9 in 2023-12-11T19:29:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:35 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:35.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:29:35.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:29:35.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-11T19:29:35.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:29:35.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: osdmap e59: 9 total, 8 up, 9 in 2023-12-11T19:29:35.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:35.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:35 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:35.997 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] boot 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: osdmap e60: 9 total, 9 up, 9 in 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:36.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:36 smithi167 ceph-mon[102750]: osdmap e61: 9 total, 9 up, 9 in 2023-12-11T19:29:36.800 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] boot 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: osdmap e60: 9 total, 9 up, 9 in 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:36.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:36.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:36 smithi066 ceph-mon[97920]: osdmap e61: 9 total, 9 up, 9 in 2023-12-11T19:29:36.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: osd.8 [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] boot 2023-12-11T19:29:36.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: osdmap e60: 9 total, 9 up, 9 in 2023-12-11T19:29:36.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T19:29:36.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:36.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:36 smithi070 ceph-mon[104044]: osdmap e61: 9 total, 9 up, 9 in 2023-12-11T19:29:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:37 smithi066 ceph-mon[97920]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:37 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi167 2023-12-11T19:29:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:37 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi167 to 5757M 2023-12-11T19:29:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:37 smithi066 ceph-mon[97920]: osdmap e62: 9 total, 9 up, 9 in 2023-12-11T19:29:37.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:37 smithi070 ceph-mon[104044]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:37.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:37 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi167 2023-12-11T19:29:37.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:37 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi167 to 5757M 2023-12-11T19:29:37.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:37 smithi070 ceph-mon[104044]: osdmap e62: 9 total, 9 up, 9 in 2023-12-11T19:29:38.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:37 smithi167 ceph-mon[102750]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:38.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:37 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi167 2023-12-11T19:29:38.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:37 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi167 to 5757M 2023-12-11T19:29:38.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:37 smithi167 ceph-mon[102750]: osdmap e62: 9 total, 9 up, 9 in 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 -- 172.21.15.66:0/1430096259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc1031c0 msgr2=0x7f27cc109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 --2- 172.21.15.66:0/1430096259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc1031c0 0x7f27cc109a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27c400b0d0 tx=0x7f27c400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 -- 172.21.15.66:0/1430096259 shutdown_connections 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 --2- 172.21.15.66:0/1430096259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc1031c0 0x7f27cc109a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 --2- 172.21.15.66:0/1430096259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc101620 0x7f27cc101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 --2- 172.21.15.66:0/1430096259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc102820 0x7f27cc102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 -- 172.21.15.66:0/1430096259 >> 172.21.15.66:0/1430096259 conn(0x7f27cc0fcbe0 msgr2=0x7f27cc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 -- 172.21.15.66:0/1430096259 shutdown_connections 2023-12-11T19:29:38.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.287+0000 7f27d3107700 1 -- 172.21.15.66:0/1430096259 wait complete. 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.288+0000 7f27d3107700 1 Processor -- start 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 -- start start 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 0x7f27cc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc102820 0x7f27cc199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc1031c0 0x7f27cc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1a11c0 con 0x7f27cc101620 2023-12-11T19:29:38.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1a1300 con 0x7f27cc1031c0 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d3107700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1a1490 con 0x7f27cc102820 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d0ea3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 0x7f27cc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d0ea3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 0x7f27cc1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53674/0 (socket says 172.21.15.66:53674) 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d0ea3700 1 -- 172.21.15.66:0/2614619022 learned_addr learned my addr 172.21.15.66:0/2614619022 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.289+0000 7f27d16a4700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc1031c0 0x7f27cc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27cbfff700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc102820 0x7f27cc199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d0ea3700 1 -- 172.21.15.66:0/2614619022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc102820 msgr2=0x7f27cc199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d0ea3700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc102820 0x7f27cc199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:38.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d0ea3700 1 -- 172.21.15.66:0/2614619022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc1031c0 msgr2=0x7f27cc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d0ea3700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc1031c0 0x7f27cc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d0ea3700 1 -- 172.21.15.66:0/2614619022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c4009d00 con 0x7f27cc101620 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d16a4700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc1031c0 0x7f27cc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d0ea3700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 0x7f27cc1996d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c9c0 tx=0x7f27c000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0007ac0 con 0x7f27cc101620 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27c0007c20 con 0x7f27cc101620 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc1a1770 con 0x7f27cc101620 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0018660 con 0x7f27cc101620 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.290+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc1a1cc0 con 0x7f27cc101620 2023-12-11T19:29:38.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.292+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc04f290 con 0x7f27cc101620 2023-12-11T19:29:38.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.294+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f27c000f450 con 0x7f27cc101620 2023-12-11T19:29:38.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.295+0000 7f27c9ffb700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27bc0a1500 0x7f27bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:29:38.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.295+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f27c00cc420 con 0x7f27cc101620 2023-12-11T19:29:38.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.295+0000 7f27cbfff700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27bc0a1500 0x7f27bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:29:38.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.296+0000 7f27cbfff700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27bc0a1500 0x7f27bc0a39c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005ea0 tx=0x7f27b8005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:29:38.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.298+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f27c0096480 con 0x7f27cc101620 2023-12-11T19:29:38.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:38.519+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f27cc074f20 con 0x7f27bc0a1500 2023-12-11T19:29:38.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:38 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:29:38.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:38 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:29:38.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:38 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:38.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:38 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:29:38.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:38 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:29:38.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:38 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:39.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:38 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:29:39.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:38 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:29:39.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:38 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:39.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:39 smithi066 ceph-mon[97920]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:39.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:39 smithi066 ceph-mon[97920]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:29:39.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:39 smithi066 ceph-mon[97920]: osdmap e63: 9 total, 9 up, 9 in 2023-12-11T19:29:39.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:39 smithi070 ceph-mon[104044]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:39.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:39 smithi070 ceph-mon[104044]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:29:39.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:39 smithi070 ceph-mon[104044]: osdmap e63: 9 total, 9 up, 9 in 2023-12-11T19:29:40.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:39 smithi167 ceph-mon[102750]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:40.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:39 smithi167 ceph-mon[102750]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:29:40.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:39 smithi167 ceph-mon[102750]: osdmap e63: 9 total, 9 up, 9 in 2023-12-11T19:29:40.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:40 smithi066 ceph-mon[97920]: osdmap e64: 9 total, 9 up, 9 in 2023-12-11T19:29:40.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:40 smithi070 ceph-mon[104044]: osdmap e64: 9 total, 9 up, 9 in 2023-12-11T19:29:41.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:40 smithi167 ceph-mon[102750]: osdmap e64: 9 total, 9 up, 9 in 2023-12-11T19:29:41.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:41 smithi066 ceph-mon[97920]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:29:41.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:41 smithi070 ceph-mon[104044]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:29:42.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:41 smithi167 ceph-mon[102750]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:29:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:42 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3440265507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "241627a2-87be-4277-9306-48d9f64595a9"}]: dispatch 2023-12-11T19:29:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:42 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3440265507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "241627a2-87be-4277-9306-48d9f64595a9"}]': finished 2023-12-11T19:29:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:42 smithi066 ceph-mon[97920]: osdmap e65: 10 total, 9 up, 10 in 2023-12-11T19:29:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:42 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:29:42.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:42 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3682970800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:29:42.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:42 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3440265507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "241627a2-87be-4277-9306-48d9f64595a9"}]: dispatch 2023-12-11T19:29:42.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:42 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3440265507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "241627a2-87be-4277-9306-48d9f64595a9"}]': finished 2023-12-11T19:29:42.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:42 smithi070 ceph-mon[104044]: osdmap e65: 10 total, 9 up, 10 in 2023-12-11T19:29:42.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:42 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:29:42.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:42 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3682970800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:29:43.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:42 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3440265507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "241627a2-87be-4277-9306-48d9f64595a9"}]: dispatch 2023-12-11T19:29:43.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:42 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3440265507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "241627a2-87be-4277-9306-48d9f64595a9"}]': finished 2023-12-11T19:29:43.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:42 smithi167 ceph-mon[102750]: osdmap e65: 10 total, 9 up, 10 in 2023-12-11T19:29:43.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:42 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:29:43.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:42 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3682970800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:29:44.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:43 smithi167 ceph-mon[102750]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2023-12-11T19:29:44.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:43 smithi066 ceph-mon[97920]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2023-12-11T19:29:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:43 smithi070 ceph-mon[104044]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 120 KiB/s, 0 objects/s recovering 2023-12-11T19:29:46.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:45 smithi167 ceph-mon[102750]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:29:46.035 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:45 smithi066 ceph-mon[97920]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:29:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:45 smithi070 ceph-mon[104044]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:29:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:47 smithi066 ceph-mon[97920]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-11T19:29:48.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:47 smithi167 ceph-mon[102750]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-11T19:29:48.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:47 smithi070 ceph-mon[104044]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-11T19:29:50.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:49 smithi167 ceph-mon[102750]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-11T19:29:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:49 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:49 smithi167 ceph-mon[102750]: Deploying daemon osd.9 on smithi066 2023-12-11T19:29:50.056 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:49 smithi066 ceph-mon[97920]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:50.057 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-11T19:29:50.057 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:49 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:50.057 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:49 smithi066 ceph-mon[97920]: Deploying daemon osd.9 on smithi066 2023-12-11T19:29:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:49 smithi070 ceph-mon[104044]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-11T19:29:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:49 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:29:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:49 smithi070 ceph-mon[104044]: Deploying daemon osd.9 on smithi066 2023-12-11T19:29:51.953 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:51 smithi066 ceph-mon[97920]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:52.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:51 smithi167 ceph-mon[102750]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:52.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:51 smithi070 ceph-mon[104044]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:54.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:53 smithi167 ceph-mon[102750]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:54.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:53 smithi066 ceph-mon[97920]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:54.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:53 smithi070 ceph-mon[104044]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:55 smithi167 ceph-mon[102750]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:55 smithi066 ceph-mon[97920]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:55 smithi070 ceph-mon[104044]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:57.100 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:57.100 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:57.100 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:57.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:57.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:57.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:57.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:57.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:29:57.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:29:58.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:57 smithi066 ceph-mon[97920]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:58.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:57 smithi070 ceph-mon[104044]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:58.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:57 smithi167 ceph-mon[102750]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:29:59.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.580+0000 7f27c9ffb700 1 -- 172.21.15.66:0/2614619022 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f27cc074f20 con 0x7f27bc0a1500 2023-12-11T19:29:59.581 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 9 on host 'smithi066' 2023-12-11T19:29:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27bc0a1500 msgr2=0x7f27bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27bc0a1500 0x7f27bc0a39c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005ea0 tx=0x7f27b8005e30 comp rx=0 tx=0).stop 2023-12-11T19:29:59.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 msgr2=0x7f27cc1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 0x7f27cc1996d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c9c0 tx=0x7f27c000cd80 comp rx=0 tx=0).stop 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 shutdown_connections 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27bc0a1500 0x7f27bc0a39c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27cc1031c0 0x7f27cc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27cc101620 0x7f27cc1996d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 --2- 172.21.15.66:0/2614619022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc102820 0x7f27cc199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 >> 172.21.15.66:0/2614619022 conn(0x7f27cc0fcbe0 msgr2=0x7f27cc073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 shutdown_connections 2023-12-11T19:29:59.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:29:59.583+0000 7f27d3107700 1 -- 172.21.15.66:0/2614619022 wait complete. 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:00.010 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:29:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.051 DEBUG:teuthology.orchestra.run.smithi066:osd.9> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.9.service 2023-12-11T19:30:00.058 INFO:tasks.cephadm:Deploying osd.10 on smithi070 with /dev/vg_nvme/lv_1... 2023-12-11T19:30:00.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-11T19:30:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:29:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.132 INFO:journalctl@ceph.osd.9.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:00.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:29:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:30:01.048 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:00 smithi066 ceph-mon[97920]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-11T19:30:01.049 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:00 smithi066 ceph-mon[97920]: overall HEALTH_OK 2023-12-11T19:30:01.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:00 smithi070 ceph-mon[104044]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-11T19:30:01.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:00 smithi070 ceph-mon[104044]: overall HEALTH_OK 2023-12-11T19:30:01.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:00 smithi167 ceph-mon[102750]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-11T19:30:01.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:00 smithi167 ceph-mon[102750]: overall HEALTH_OK 2023-12-11T19:30:02.063 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:30:01.853+0000 7fdb98d2c700 -1 osd.9 0 waiting for initial osdmap 2023-12-11T19:30:02.063 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:30:01.878+0000 7fdb90b1e700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:30:02.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-mon[97920]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:02.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-mon[97920]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-11T19:30:02.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-mon[97920]: osdmap e66: 10 total, 9 up, 10 in 2023-12-11T19:30:02.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-mon[97920]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:30:02.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:01 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:01 smithi070 ceph-mon[104044]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:01 smithi070 ceph-mon[104044]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-11T19:30:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:01 smithi070 ceph-mon[104044]: osdmap e66: 10 total, 9 up, 10 in 2023-12-11T19:30:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:01 smithi070 ceph-mon[104044]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:30:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:01 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:01 smithi167 ceph-mon[102750]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:01 smithi167 ceph-mon[102750]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-11T19:30:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:01 smithi167 ceph-mon[102750]: osdmap e66: 10 total, 9 up, 10 in 2023-12-11T19:30:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:01 smithi167 ceph-mon[102750]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-12-11T19:30:02.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:01 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-11T19:30:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-11T19:30:03.026 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2023-12-11T19:30:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2023-12-11T19:30:03.026 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0322264 s, 325 MB/s 2023-12-11T19:30:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2023-12-11T19:30:03.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:02 smithi066 ceph-mon[97920]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:30:03.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:02 smithi066 ceph-mon[97920]: osdmap e67: 10 total, 9 up, 10 in 2023-12-11T19:30:03.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:02 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:02 smithi070 ceph-mon[104044]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:30:03.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:02 smithi070 ceph-mon[104044]: osdmap e67: 10 total, 9 up, 10 in 2023-12-11T19:30:03.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:02 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:02 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:02 smithi167 ceph-mon[102750]: from='osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-12-11T19:30:03.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:02 smithi167 ceph-mon[102750]: osdmap e67: 10 total, 9 up, 10 in 2023-12-11T19:30:03.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:02 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:03.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_1 2023-12-11T19:30:04.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:30:04.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:30:04.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:04.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] boot 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: osdmap e68: 10 total, 10 up, 10 in 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:04.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:03 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:30:04.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] boot 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: osdmap e68: 10 total, 10 up, 10 in 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:04.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:03 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.213 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: osd.9 [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] boot 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: osdmap e68: 10 total, 10 up, 10 in 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:04.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:03 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:04 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi066 2023-12-11T19:30:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:04 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi066 to 4317M 2023-12-11T19:30:05.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:04 smithi070 ceph-mon[104044]: osdmap e69: 10 total, 10 up, 10 in 2023-12-11T19:30:05.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:04 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi066 2023-12-11T19:30:05.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:04 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi066 to 4317M 2023-12-11T19:30:05.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:04 smithi167 ceph-mon[102750]: osdmap e69: 10 total, 10 up, 10 in 2023-12-11T19:30:05.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:04 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi066 2023-12-11T19:30:05.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:04 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi066 to 4317M 2023-12-11T19:30:05.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:04 smithi066 ceph-mon[97920]: osdmap e69: 10 total, 10 up, 10 in 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 -- 172.21.15.70:0/3526955051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34101620 msgr2=0x7f1c34101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/3526955051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34101620 0x7f1c34101a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1c300099c0 tx=0x7f1c30009cd0 comp rx=0 tx=0).stop 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 -- 172.21.15.70:0/3526955051 shutdown_connections 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/3526955051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34101620 0x7f1c34101a40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/3526955051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c341031c0 0x7f1c34109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/3526955051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34102820 0x7f1c34102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 -- 172.21.15.70:0/3526955051 >> 172.21.15.70:0/3526955051 conn(0x7f1c340fcbe0 msgr2=0x7f1c340ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 -- 172.21.15.70:0/3526955051 shutdown_connections 2023-12-11T19:30:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.742+0000 7f1c3c10b700 1 -- 172.21.15.70:0/3526955051 wait complete. 2023-12-11T19:30:05.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.743+0000 7f1c3c10b700 1 Processor -- start 2023-12-11T19:30:05.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 -- start start 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34101620 0x7f1c34199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 0x7f1c34199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c341031c0 0x7f1c341a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c341a1230 con 0x7f1c34101620 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c341a1370 con 0x7f1c34102820 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c3c10b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c341a1500 con 0x7f1c341031c0 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c396a6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 0x7f1c34199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.744+0000 7f1c39ea7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34101620 0x7f1c34199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c396a6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 0x7f1c34199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48448/0 (socket says 172.21.15.70:48448) 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c396a6700 1 -- 172.21.15.70:0/1372036957 learned_addr learned my addr 172.21.15.70:0/1372036957 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c3a6a8700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c341031c0 0x7f1c341a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c39ea7700 1 -- 172.21.15.70:0/1372036957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c341031c0 msgr2=0x7f1c341a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c39ea7700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c341031c0 0x7f1c341a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c39ea7700 1 -- 172.21.15.70:0/1372036957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 msgr2=0x7f1c34199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c39ea7700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 0x7f1c34199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c39ea7700 1 -- 172.21.15.70:0/1372036957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300095f0 con 0x7f1c34101620 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c396a6700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 0x7f1c34199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c39ea7700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34101620 0x7f1c34199790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009fd0 tx=0x7f1c3000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.745+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3001d070 con 0x7f1c34101620 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.746+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c341a1780 con 0x7f1c34101620 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.746+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c3000bd30 con 0x7f1c34101620 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.746+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300173f0 con 0x7f1c34101620 2023-12-11T19:30:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.746+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c341a1bf0 con 0x7f1c34101620 2023-12-11T19:30:05.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.747+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3404f290 con 0x7f1c34101620 2023-12-11T19:30:05.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.749+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1c30022470 con 0x7f1c34101620 2023-12-11T19:30:05.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.750+0000 7f1c2affd700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1c200a1580 0x7f1c200a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:05.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.750+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1c300cdb20 con 0x7f1c34101620 2023-12-11T19:30:05.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.751+0000 7f1c396a6700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1c200a1580 0x7f1c200a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:05.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.751+0000 7f1c396a6700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1c200a1580 0x7f1c200a3a40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c24011550 tx=0x7f1c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:30:05.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.753+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1c30097970 con 0x7f1c34101620 2023-12-11T19:30:05.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:05.967+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c34109780 con 0x7f1c200a1580 2023-12-11T19:30:06.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:05 smithi167 ceph-mon[102750]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:06.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:05 smithi066 ceph-mon[97920]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:06.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:05 smithi070 ceph-mon[104044]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:07.142 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:06 smithi070 ceph-mon[104044]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:30:07.142 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:30:07.143 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:30:07.143 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:07.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:06 smithi167 ceph-mon[102750]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:30:07.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:30:07.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:30:07.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:07.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:06 smithi066 ceph-mon[97920]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:30:07.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:30:07.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:30:07.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:08.070 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:07 smithi070 ceph-mon[104044]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:08.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:07 smithi167 ceph-mon[102750]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:08.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:07 smithi066 ceph-mon[97920]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:10.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:09 smithi167 ceph-mon[102750]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:10.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:09 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/3269881089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]: dispatch 2023-12-11T19:30:10.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:09 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]: dispatch 2023-12-11T19:30:10.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:09 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]': finished 2023-12-11T19:30:10.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:09 smithi167 ceph-mon[102750]: osdmap e70: 11 total, 10 up, 11 in 2023-12-11T19:30:10.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:09 smithi066 ceph-mon[97920]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:09 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/3269881089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]: dispatch 2023-12-11T19:30:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:09 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]: dispatch 2023-12-11T19:30:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:09 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]': finished 2023-12-11T19:30:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:09 smithi066 ceph-mon[97920]: osdmap e70: 11 total, 10 up, 11 in 2023-12-11T19:30:10.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:10.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:09 smithi070 ceph-mon[104044]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:10.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/3269881089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]: dispatch 2023-12-11T19:30:10.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:09 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]: dispatch 2023-12-11T19:30:10.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:09 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afbe1b4e-b374-46a0-884f-89c5a4cf84ab"}]': finished 2023-12-11T19:30:10.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:09 smithi070 ceph-mon[104044]: osdmap e70: 11 total, 10 up, 11 in 2023-12-11T19:30:10.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:11.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:10 smithi167 ceph-mon[102750]: from='client.? 172.21.15.70:0/3300342513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:30:11.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:10 smithi066 ceph-mon[97920]: from='client.? 172.21.15.70:0/3300342513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:30:11.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:10 smithi070 ceph-mon[104044]: from='client.? 172.21.15.70:0/3300342513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:30:12.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:11 smithi167 ceph-mon[102750]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:12.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:11 smithi066 ceph-mon[97920]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:12.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:11 smithi070 ceph-mon[104044]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:14.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:13 smithi167 ceph-mon[102750]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:14.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:13 smithi066 ceph-mon[97920]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:14.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:13 smithi070 ceph-mon[104044]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:16.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:15 smithi070 ceph-mon[104044]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:16.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:15 smithi167 ceph-mon[102750]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:16.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:15 smithi066 ceph-mon[97920]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:17.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:17 smithi070 ceph-mon[104044]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:17.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-11T19:30:17.742 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:17.743 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:17 smithi070 ceph-mon[104044]: Deploying daemon osd.10 on smithi070 2023-12-11T19:30:17.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:17 smithi167 ceph-mon[102750]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-11T19:30:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:17 smithi167 ceph-mon[102750]: Deploying daemon osd.10 on smithi070 2023-12-11T19:30:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:17 smithi066 ceph-mon[97920]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-11T19:30:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:17 smithi066 ceph-mon[97920]: Deploying daemon osd.10 on smithi070 2023-12-11T19:30:19.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:19 smithi070 ceph-mon[104044]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:19.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:19 smithi167 ceph-mon[102750]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:19.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:19 smithi066 ceph-mon[97920]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:21.720 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:21 smithi070 ceph-mon[104044]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:21.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:21 smithi167 ceph-mon[102750]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:21.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:21 smithi066 ceph-mon[97920]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:24 smithi070 ceph-mon[104044]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:24.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:24.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:24 smithi167 ceph-mon[102750]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:24.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:24.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:24.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:24.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:24 smithi066 ceph-mon[97920]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:26.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:26 smithi167 ceph-mon[102750]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:26.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:26 smithi066 ceph-mon[97920]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:26.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:26 smithi070 ceph-mon[104044]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:26.744 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 10 on host 'smithi070' 2023-12-11T19:30:26.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.742+0000 7f1c2affd700 1 -- 172.21.15.70:0/1372036957 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1c34109780 con 0x7f1c200a1580 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1c200a1580 msgr2=0x7f1c200a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1c200a1580 0x7f1c200a3a40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c24011550 tx=0x7f1c24009260 comp rx=0 tx=0).stop 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34101620 msgr2=0x7f1c34199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34101620 0x7f1c34199790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009fd0 tx=0x7f1c3000fa00 comp rx=0 tx=0).stop 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 shutdown_connections 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1c200a1580 0x7f1c200a3a40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c341031c0 0x7f1c341a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c34101620 0x7f1c34199790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 --2- 172.21.15.70:0/1372036957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c34102820 0x7f1c34199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 >> 172.21.15.70:0/1372036957 conn(0x7f1c340fcbe0 msgr2=0x7f1c34108060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 shutdown_connections 2023-12-11T19:30:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-12-11T19:30:26.745+0000 7f1c3c10b700 1 -- 172.21.15.70:0/1372036957 wait complete. 2023-12-11T19:30:27.257 DEBUG:teuthology.orchestra.run.smithi070:osd.10> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.10.service 2023-12-11T19:30:27.262 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2023-12-11T19:30:27.262 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-11T19:30:27.334 INFO:journalctl@ceph.osd.10.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:30:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T19:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:27 smithi167 ceph-mon[102750]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T19:30:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T19:30:27.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:27 smithi066 ceph-mon[97920]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T19:30:27.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:27.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:27.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T19:30:27.863 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:27 smithi070 ceph-mon[104044]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T19:30:28.054 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:30:29.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:29 smithi167 ceph-mon[102750]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-11T19:30:29.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:29 smithi167 ceph-mon[102750]: from='osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:30:29.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:29 smithi167 ceph-mon[102750]: osdmap e71: 11 total, 10 up, 11 in 2023-12-11T19:30:29.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:29 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:29.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:29 smithi167 ceph-mon[102750]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:30:29.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-mon[104044]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-11T19:30:29.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-mon[104044]: from='osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:30:29.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-mon[104044]: osdmap e71: 11 total, 10 up, 11 in 2023-12-11T19:30:29.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:29.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-mon[104044]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:30:29.361 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:30:29.140+0000 7f4a52134700 -1 osd.10 0 waiting for initial osdmap 2023-12-11T19:30:29.361 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:30:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:30:29.156+0000 7f4a49f26700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:30:29.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:29 smithi066 ceph-mon[97920]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-11T19:30:29.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:29 smithi066 ceph-mon[97920]: from='osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:30:29.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:29 smithi066 ceph-mon[97920]: osdmap e71: 11 total, 10 up, 11 in 2023-12-11T19:30:29.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:29 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:29.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:29 smithi066 ceph-mon[97920]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2023-12-11T19:30:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-11T19:30:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-11T19:30:30.160 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-12-11T19:30:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-12-11T19:30:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0547319 s, 192 MB/s 2023-12-11T19:30:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: osdmap e72: 11 total, 10 up, 11 in 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] boot 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: osdmap e73: 11 total, 11 up, 11 in 2023-12-11T19:30:30.461 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: osdmap e72: 11 total, 10 up, 11 in 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] boot 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: osdmap e73: 11 total, 11 up, 11 in 2023-12-11T19:30:30.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: osdmap e72: 11 total, 10 up, 11 in 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: osd.10 [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] boot 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: osdmap e73: 11 total, 11 up, 11 in 2023-12-11T19:30:30.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T19:30:30.638 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2023-12-11T19:30:31.419 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi070 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi070 to 4317M 2023-12-11T19:30:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:31 smithi167 ceph-mon[102750]: osdmap e74: 11 total, 11 up, 11 in 2023-12-11T19:30:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi070 2023-12-11T19:30:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi070 to 4317M 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:31 smithi066 ceph-mon[97920]: osdmap e74: 11 total, 11 up, 11 in 2023-12-11T19:30:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi070 2023-12-11T19:30:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi070 to 4317M 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:32.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:31 smithi070 ceph-mon[104044]: osdmap e74: 11 total, 11 up, 11 in 2023-12-11T19:30:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 -- 172.21.15.167:0/107578933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c1031c0 msgr2=0x7f313c109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 --2- 172.21.15.167:0/107578933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c1031c0 0x7f313c109a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f313400b0a0 tx=0x7f313400b3b0 comp rx=0 tx=0).stop 2023-12-11T19:30:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 -- 172.21.15.167:0/107578933 shutdown_connections 2023-12-11T19:30:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 --2- 172.21.15.167:0/107578933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f313c101620 0x7f313c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 --2- 172.21.15.167:0/107578933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c1031c0 0x7f313c109a70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 --2- 172.21.15.167:0/107578933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c102820 0x7f313c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:32.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 -- 172.21.15.167:0/107578933 >> 172.21.15.167:0/107578933 conn(0x7f313c0fcbe0 msgr2=0x7f313c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:30:32.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 -- 172.21.15.167:0/107578933 shutdown_connections 2023-12-11T19:30:32.928 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.925+0000 7f3143237700 1 -- 172.21.15.167:0/107578933 wait complete. 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.926+0000 7f3143237700 1 Processor -- start 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.926+0000 7f3143237700 1 -- start start 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.926+0000 7f3143237700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 0x7f313c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3143237700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f313c102820 0x7f313c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3143237700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c1031c0 0x7f313c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3143237700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1a1280 con 0x7f313c1031c0 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3143237700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1a13c0 con 0x7f313c102820 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3143237700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1a1550 con 0x7f313c101620 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 0x7f313c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 0x7f313c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:41710/0 (socket says 172.21.15.167:41710) 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 -- 172.21.15.167:0/3286541186 learned_addr learned my addr 172.21.15.167:0/3286541186 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f313bfff700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f313c102820 0x7f313c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f31417d4700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c1031c0 0x7f313c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 -- 172.21.15.167:0/3286541186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f313c102820 msgr2=0x7f313c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f313c102820 0x7f313c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 -- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c1031c0 msgr2=0x7f313c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c1031c0 0x7f313c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f3140fd3700 1 -- 172.21.15.167:0/3286541186 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134009d00 con 0x7f313c101620 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.927+0000 7f31417d4700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c1031c0 0x7f313c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.928+0000 7f3140fd3700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 0x7f313c199740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f313000d9b0 tx=0x7f313000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.928+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130004020 con 0x7f313c101620 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.928+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3130009730 con 0x7f313c101620 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.928+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c1a1830 con 0x7f313c101620 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.928+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300039c0 con 0x7f313c101620 2023-12-11T19:30:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.928+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c1a1d80 con 0x7f313c101620 2023-12-11T19:30:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.929+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c04f290 con 0x7f313c101620 2023-12-11T19:30:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.931+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f31300098a0 con 0x7f313c101620 2023-12-11T19:30:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.932+0000 7f3139ffb700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f312c0a1500 0x7f312c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.932+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f31300cc3e0 con 0x7f313c101620 2023-12-11T19:30:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.933+0000 7f313bfff700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f312c0a1500 0x7f312c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.934+0000 7f313bfff700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f312c0a1500 0x7f312c0a39c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3128009c40 tx=0x7f3128009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:30:32.938 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:32.937+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3130095fd0 con 0x7f313c101620 2023-12-11T19:30:33.149 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:33.147+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f313c074f20 con 0x7f312c0a1500 2023-12-11T19:30:33.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:33 smithi167 ceph-mon[102750]: osdmap e75: 11 total, 11 up, 11 in 2023-12-11T19:30:33.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:33 smithi066 ceph-mon[97920]: osdmap e75: 11 total, 11 up, 11 in 2023-12-11T19:30:33.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:33 smithi070 ceph-mon[104044]: osdmap e75: 11 total, 11 up, 11 in 2023-12-11T19:30:34.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:34 smithi167 ceph-mon[102750]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:34.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:34 smithi167 ceph-mon[102750]: osdmap e76: 11 total, 11 up, 11 in 2023-12-11T19:30:34.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:34 smithi167 ceph-mon[102750]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:30:34.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:30:34.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:30:34.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:34 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:34 smithi066 ceph-mon[97920]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:34 smithi066 ceph-mon[97920]: osdmap e76: 11 total, 11 up, 11 in 2023-12-11T19:30:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:34 smithi066 ceph-mon[97920]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:30:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:30:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:30:34.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:34 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:34.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:34 smithi070 ceph-mon[104044]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:34.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:34 smithi070 ceph-mon[104044]: osdmap e76: 11 total, 11 up, 11 in 2023-12-11T19:30:34.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:34 smithi070 ceph-mon[104044]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:30:34.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T19:30:34.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T19:30:34.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:34 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:35.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:35 smithi167 ceph-mon[102750]: osdmap e77: 11 total, 11 up, 11 in 2023-12-11T19:30:35.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:35 smithi066 ceph-mon[97920]: osdmap e77: 11 total, 11 up, 11 in 2023-12-11T19:30:35.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:35 smithi070 ceph-mon[104044]: osdmap e77: 11 total, 11 up, 11 in 2023-12-11T19:30:36.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:36 smithi167 ceph-mon[102750]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:30:36.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:36 smithi066 ceph-mon[97920]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:30:36.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:36 smithi070 ceph-mon[104044]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T19:30:37.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:37 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/111181642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]: dispatch 2023-12-11T19:30:37.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:37 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]: dispatch 2023-12-11T19:30:37.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:37 smithi167 ceph-mon[102750]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]': finished 2023-12-11T19:30:37.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:37 smithi167 ceph-mon[102750]: osdmap e78: 12 total, 11 up, 12 in 2023-12-11T19:30:37.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:37.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:37 smithi167 ceph-mon[102750]: from='client.? 172.21.15.167:0/2705121054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:30:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:37 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/111181642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]: dispatch 2023-12-11T19:30:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:37 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]: dispatch 2023-12-11T19:30:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:37 smithi066 ceph-mon[97920]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]': finished 2023-12-11T19:30:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:37 smithi066 ceph-mon[97920]: osdmap e78: 12 total, 11 up, 12 in 2023-12-11T19:30:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:37 smithi066 ceph-mon[97920]: from='client.? 172.21.15.167:0/2705121054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:30:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:37 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/111181642' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]: dispatch 2023-12-11T19:30:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:37 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]: dispatch 2023-12-11T19:30:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:37 smithi070 ceph-mon[104044]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af4e5d83-929b-49e5-af0e-9e2524e9d264"}]': finished 2023-12-11T19:30:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:37 smithi070 ceph-mon[104044]: osdmap e78: 12 total, 11 up, 12 in 2023-12-11T19:30:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:37 smithi070 ceph-mon[104044]: from='client.? 172.21.15.167:0/2705121054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T19:30:38.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:38 smithi167 ceph-mon[102750]: pgmap v260: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-12-11T19:30:38.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:38 smithi066 ceph-mon[97920]: pgmap v260: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-12-11T19:30:38.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:38 smithi070 ceph-mon[104044]: pgmap v260: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2023-12-11T19:30:40.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:40 smithi167 ceph-mon[102750]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:30:40.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:40 smithi066 ceph-mon[97920]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:30:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:40 smithi070 ceph-mon[104044]: pgmap v262: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T19:30:42.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:42 smithi167 ceph-mon[102750]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-12-11T19:30:42.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:42 smithi066 ceph-mon[97920]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-12-11T19:30:42.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:42 smithi070 ceph-mon[104044]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2023-12-11T19:30:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:44 smithi167 ceph-mon[102750]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-11T19:30:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:44 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:44.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:44 smithi066 ceph-mon[97920]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:44.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-11T19:30:44.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:44 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:44.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:44 smithi070 ceph-mon[104044]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:44.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-11T19:30:44.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:44 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:45.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:45 smithi167 ceph-mon[102750]: Deploying daemon osd.11 on smithi167 2023-12-11T19:30:45.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:45 smithi066 ceph-mon[97920]: Deploying daemon osd.11 on smithi167 2023-12-11T19:30:45.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:45 smithi070 ceph-mon[104044]: Deploying daemon osd.11 on smithi167 2023-12-11T19:30:46.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:46 smithi167 ceph-mon[102750]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:46.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:46 smithi066 ceph-mon[97920]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:46.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:46 smithi070 ceph-mon[104044]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:47.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:47 smithi167 ceph-mon[102750]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:47.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:47 smithi066 ceph-mon[97920]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:47.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:47 smithi070 ceph-mon[104044]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:49.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:49 smithi167 ceph-mon[102750]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:49.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:49 smithi066 ceph-mon[97920]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:49.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:49 smithi070 ceph-mon[104044]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:51 smithi167 ceph-mon[102750]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:51.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:51 smithi066 ceph-mon[97920]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:51.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:51 smithi070 ceph-mon[104044]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:51.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:53.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:53 smithi167 ceph-mon[102750]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:53.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:53 smithi066 ceph-mon[97920]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:53.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:53 smithi070 ceph-mon[104044]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:54.329 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2023-12-11T19:30:54.329 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.326+0000 7f3139ffb700 1 -- 172.21.15.167:0/3286541186 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f313c074f20 con 0x7f312c0a1500 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f312c0a1500 msgr2=0x7f312c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f312c0a1500 0x7f312c0a39c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3128009c40 tx=0x7f3128009260 comp rx=0 tx=0).stop 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 msgr2=0x7f313c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 0x7f313c199740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f313000d9b0 tx=0x7f313000dcc0 comp rx=0 tx=0).stop 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 shutdown_connections 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f312c0a1500 0x7f312c0a39c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f313c1031c0 0x7f313c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f313c101620 0x7f313c199740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 --2- 172.21.15.167:0/3286541186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f313c102820 0x7f313c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 >> 172.21.15.167:0/3286541186 conn(0x7f313c0fcbe0 msgr2=0x7f313c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 shutdown_connections 2023-12-11T19:30:54.331 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:30:54.329+0000 7f3143237700 1 -- 172.21.15.167:0/3286541186 wait complete. 2023-12-11T19:30:54.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T19:30:54.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:54 smithi167 ceph-mon[102750]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T19:30:54.851 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.11.service 2023-12-11T19:30:54.857 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-11T19:30:54.857 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd stat -f json 2023-12-11T19:30:54.930 INFO:journalctl@ceph.osd.11.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:30:54.932 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.932 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.932 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T19:30:54.933 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:54 smithi066 ceph-mon[97920]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T19:30:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:55.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:55.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:30:55.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T19:30:55.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:54 smithi070 ceph-mon[104044]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T19:30:55.662 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:30:55.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:55 smithi167 ceph-mon[102750]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:55.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:55 smithi167 ceph-mon[102750]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T19:30:55.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:55 smithi167 ceph-mon[102750]: osdmap e79: 12 total, 11 up, 12 in 2023-12-11T19:30:55.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:55.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:55 smithi167 ceph-mon[102750]: from='osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:30:55.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:55 smithi167 ceph-mon[102750]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:30:55.962 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:55 smithi066 ceph-mon[97920]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:55.962 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:55 smithi066 ceph-mon[97920]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T19:30:55.962 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:55 smithi066 ceph-mon[97920]: osdmap e79: 12 total, 11 up, 12 in 2023-12-11T19:30:55.963 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:55.963 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:55 smithi066 ceph-mon[97920]: from='osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:30:55.963 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:55 smithi066 ceph-mon[97920]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:30:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:55 smithi070 ceph-mon[104044]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:55 smithi070 ceph-mon[104044]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T19:30:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:55 smithi070 ceph-mon[104044]: osdmap e79: 12 total, 11 up, 12 in 2023-12-11T19:30:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:55 smithi070 ceph-mon[104044]: from='osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:30:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:55 smithi070 ceph-mon[104044]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-12-11T19:30:56.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:30:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:30:56.336+0000 7fbd7dcb3700 -1 osd.11 0 waiting for initial osdmap 2023-12-11T19:30:56.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:30:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:30:56.353+0000 7fbd76aa7700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.256+0000 7f597467d700 1 -- 172.21.15.66:0/2757703943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c102820 msgr2=0x7f596c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.256+0000 7f597467d700 1 --2- 172.21.15.66:0/2757703943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c102820 0x7f596c102c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f595c0099c0 tx=0x7f595c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 -- 172.21.15.66:0/2757703943 shutdown_connections 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 --2- 172.21.15.66:0/2757703943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c101620 0x7f596c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 --2- 172.21.15.66:0/2757703943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c1031c0 0x7f596c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 --2- 172.21.15.66:0/2757703943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c102820 0x7f596c102c80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 -- 172.21.15.66:0/2757703943 >> 172.21.15.66:0/2757703943 conn(0x7f596c0fcbe0 msgr2=0x7f596c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 -- 172.21.15.66:0/2757703943 shutdown_connections 2023-12-11T19:30:57.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.257+0000 7f597467d700 1 -- 172.21.15.66:0/2757703943 wait complete. 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 Processor -- start 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 -- start start 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c101620 0x7f596c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c102820 0x7f596c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c1031c0 0x7f596c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1a1240 con 0x7f596c101620 2023-12-11T19:30:57.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1a1380 con 0x7f596c1031c0 2023-12-11T19:30:57.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f597467d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1a1510 con 0x7f596c102820 2023-12-11T19:30:57.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f5972419700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c101620 0x7f596c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:57.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f5971c18700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c102820 0x7f596c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:57.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.258+0000 7f5971c18700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c102820 0x7f596c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:35074/0 (socket says 172.21.15.66:35074) 2023-12-11T19:30:57.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5971c18700 1 -- 172.21.15.66:0/219499847 learned_addr learned my addr 172.21.15.66:0/219499847 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972419700 1 -- 172.21.15.66:0/219499847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c102820 msgr2=0x7f596c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972c1a700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c1031c0 0x7f596c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972419700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c102820 0x7f596c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972419700 1 -- 172.21.15.66:0/219499847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c1031c0 msgr2=0x7f596c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972419700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c1031c0 0x7f596c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972419700 1 -- 172.21.15.66:0/219499847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c0095f0 con 0x7f596c101620 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972c1a700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c1031c0 0x7f596c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f5972419700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c101620 0x7f596c199700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f596800d770 tx=0x7f596800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.259+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968004220 con 0x7f596c101620 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.260+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5968004380 con 0x7f596c101620 2023-12-11T19:30:57.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.260+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968003ad0 con 0x7f596c101620 2023-12-11T19:30:57.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.260+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c1a17f0 con 0x7f596c101620 2023-12-11T19:30:57.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.260+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c1a1cf0 con 0x7f596c101620 2023-12-11T19:30:57.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.261+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c04f290 con 0x7f596c101620 2023-12-11T19:30:57.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.263+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5968003c30 con 0x7f596c101620 2023-12-11T19:30:57.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.264+0000 7f59637fe700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f59580a1500 0x7f59580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:30:57.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.264+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f59680cbed0 con 0x7f596c101620 2023-12-11T19:30:57.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.264+0000 7f5971c18700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f59580a1500 0x7f59580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:30:57.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.265+0000 7f5971c18700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f59580a1500 0x7f59580a39c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f595c006010 tx=0x7f595c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:30:57.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.269+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5968095a50 con 0x7f596c101620 2023-12-11T19:30:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.488+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f596c10cfa0 con 0x7f596c101620 2023-12-11T19:30:57.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.488+0000 7f59637fe700 1 -- 172.21.15.66:0/219499847 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5968099070 con 0x7f596c101620 2023-12-11T19:30:57.490 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f59580a1500 msgr2=0x7f59580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f59580a1500 0x7f59580a39c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f595c006010 tx=0x7f595c003680 comp rx=0 tx=0).stop 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c101620 msgr2=0x7f596c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c101620 0x7f596c199700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f596800d770 tx=0x7f596800da80 comp rx=0 tx=0).stop 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 shutdown_connections 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f59580a1500 0x7f59580a39c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f596c1031c0 0x7f596c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c101620 0x7f596c199700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 --2- 172.21.15.66:0/219499847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f596c102820 0x7f596c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 >> 172.21.15.66:0/219499847 conn(0x7f596c0fcbe0 msgr2=0x7f596c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 shutdown_connections 2023-12-11T19:30:57.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:30:57.491+0000 7f597467d700 1 -- 172.21.15.66:0/219499847 wait complete. 2023-12-11T19:30:57.608 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:57 smithi066 ceph-mon[97920]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:57.608 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:57 smithi066 ceph-mon[97920]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:30:57.608 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:57 smithi066 ceph-mon[97920]: osdmap e80: 12 total, 11 up, 12 in 2023-12-11T19:30:57.608 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:57 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:57.608 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:57 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:57.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:57 smithi070 ceph-mon[104044]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:57.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:57 smithi070 ceph-mon[104044]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:30:57.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:57 smithi070 ceph-mon[104044]: osdmap e80: 12 total, 11 up, 12 in 2023-12-11T19:30:57.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:57 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:57.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:57 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:57.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:57 smithi167 ceph-mon[102750]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T19:30:57.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:57 smithi167 ceph-mon[102750]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-12-11T19:30:57.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:57 smithi167 ceph-mon[102750]: osdmap e80: 12 total, 11 up, 12 in 2023-12-11T19:30:57.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:57 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:57.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:57 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:57.962 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1702323057,"num_in_osds":12,"osd_in_since":1702323036,"num_remapped_pgs":0} 2023-12-11T19:30:57.962 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-11T19:30:58.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:58 smithi070 ceph-mon[104044]: purged_snaps scrub starts 2023-12-11T19:30:58.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:58 smithi070 ceph-mon[104044]: purged_snaps scrub ok 2023-12-11T19:30:58.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:58 smithi070 ceph-mon[104044]: osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] boot 2023-12-11T19:30:58.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:58 smithi070 ceph-mon[104044]: osdmap e81: 12 total, 12 up, 12 in 2023-12-11T19:30:58.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:58 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:58.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:58 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/219499847' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T19:30:58.762 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:30:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:58 smithi167 ceph-mon[102750]: purged_snaps scrub starts 2023-12-11T19:30:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:58 smithi167 ceph-mon[102750]: purged_snaps scrub ok 2023-12-11T19:30:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:58 smithi167 ceph-mon[102750]: osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] boot 2023-12-11T19:30:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:58 smithi167 ceph-mon[102750]: osdmap e81: 12 total, 12 up, 12 in 2023-12-11T19:30:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:58 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:58.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:58 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/219499847' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T19:30:58.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:58 smithi066 ceph-mon[97920]: purged_snaps scrub starts 2023-12-11T19:30:58.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:58 smithi066 ceph-mon[97920]: purged_snaps scrub ok 2023-12-11T19:30:58.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:58 smithi066 ceph-mon[97920]: osd.11 [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] boot 2023-12-11T19:30:58.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:58 smithi066 ceph-mon[97920]: osdmap e81: 12 total, 12 up, 12 in 2023-12-11T19:30:58.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:58 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T19:30:58.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:58 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/219499847' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: Detected new or changed devices on smithi167 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: osdmap e82: 12 total, 12 up, 12 in 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: Adjusting osd_memory_target on smithi167 to 4317M 2023-12-11T19:30:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:59.564 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:30:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:30:59.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: Detected new or changed devices on smithi167 2023-12-11T19:30:59.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: osdmap e82: 12 total, 12 up, 12 in 2023-12-11T19:30:59.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: Adjusting osd_memory_target on smithi167 to 4317M 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:59.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:30:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: Detected new or changed devices on smithi167 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: osdmap e82: 12 total, 12 up, 12 in 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: Adjusting osd_memory_target on smithi167 to 4317M 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:30:59.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:30:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/799005250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4106940 msgr2=0x7f8eb410e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/799005250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4106940 0x7f8eb410e4c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b0a0 tx=0x7f8eac00b3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/799005250 shutdown_connections 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/799005250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8eb4101910 0x7f8eb4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/799005250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4106940 0x7f8eb410e4c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/799005250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb41022c0 0x7f8eb4106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/799005250 >> 172.21.15.66:0/799005250 conn(0x7f8eb40786f0 msgr2=0x7f8eb4078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/799005250 shutdown_connections 2023-12-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.165+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/799005250 wait complete. 2023-12-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.166+0000 7f8eb8bfc700 1 Processor -- start 2023-12-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.166+0000 7f8eb8bfc700 1 -- start start 2023-12-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb8bfc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 0x7f8eb419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb8bfc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8eb41022c0 0x7f8eb419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb8bfc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4106940 0x7f8eb4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb8bfc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb419a090 con 0x7f8eb4101910 2023-12-11T19:31:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb8bfc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb419a200 con 0x7f8eb41022c0 2023-12-11T19:31:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb8bfc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb419a3c0 con 0x7f8eb4106940 2023-12-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 0x7f8eb419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 0x7f8eb419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50924/0 (socket says 172.21.15.66:50924) 2023-12-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb259c700 1 -- 172.21.15.66:0/3401069633 learned_addr learned my addr 172.21.15.66:0/3401069633 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb2d9d700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4106940 0x7f8eb4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb1d9b700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8eb41022c0 0x7f8eb419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb259c700 1 -- 172.21.15.66:0/3401069633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4106940 msgr2=0x7f8eb4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.167+0000 7f8eb259c700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4106940 0x7f8eb4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8eb259c700 1 -- 172.21.15.66:0/3401069633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8eb41022c0 msgr2=0x7f8eb419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8eb259c700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8eb41022c0 0x7f8eb419fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8eb259c700 1 -- 172.21.15.66:0/3401069633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009d00 con 0x7f8eb4101910 2023-12-11T19:31:00.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8eb259c700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 0x7f8eb419f8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400d950 tx=0x7f8ea400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4009940 con 0x7f8eb4101910 2023-12-11T19:31:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ea4010460 con 0x7f8eb4101910 2023-12-11T19:31:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400f620 con 0x7f8eb4101910 2023-12-11T19:31:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb419a6a0 con 0x7f8eb4101910 2023-12-11T19:31:00.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.168+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb41a68e0 con 0x7f8eb4101910 2023-12-11T19:31:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.169+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb4102cd0 con 0x7f8eb4101910 2023-12-11T19:31:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.171+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8ea40105d0 con 0x7f8eb4101910 2023-12-11T19:31:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.173+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8e9c0a1500 0x7f8e9c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.173+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8ea40cc6e0 con 0x7f8eb4101910 2023-12-11T19:31:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.173+0000 7f8eb1d9b700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8e9c0a1500 0x7f8e9c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.174+0000 7f8eb1d9b700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8e9c0a1500 0x7f8e9c0a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005950 tx=0x7f8ea80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:00.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.177+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8ea4096140 con 0x7f8eb4101910 2023-12-11T19:31:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.390+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eb404f290 con 0x7f8eb4101910 2023-12-11T19:31:00.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.391+0000 7f8ea37fe700 1 -- 172.21.15.66:0/3401069633 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18314 (secure 0 0 0) 0x7f8ea4099760 con 0x7f8eb4101910 2023-12-11T19:31:00.394 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:00.394 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:30:58.331837+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8e9c0a1500 msgr2=0x7f8e9c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8e9c0a1500 0x7f8e9c0a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005950 tx=0x7f8ea80058e0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 msgr2=0x7f8eb419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 0x7f8eb419f8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400d950 tx=0x7f8ea400dc60 comp rx=0 tx=0).stop 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 shutdown_connections 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8e9c0a1500 0x7f8e9c0a39c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4106940 0x7f8eb4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb4101910 0x7f8eb419f8f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 --2- 172.21.15.66:0/3401069633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8eb41022c0 0x7f8eb419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 >> 172.21.15.66:0/3401069633 conn(0x7f8eb40786f0 msgr2=0x7f8eb40ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 shutdown_connections 2023-12-11T19:31:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:00.394+0000 7f8eb8bfc700 1 -- 172.21.15.66:0/3401069633 wait complete. 2023-12-11T19:31:00.963 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:31:00.963 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-11T19:31:01.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:01 smithi070 ceph-mon[104044]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:01.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:01 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3401069633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:01.761 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:31:01.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:01 smithi167 ceph-mon[102750]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:01.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:01 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3401069633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:01 smithi066 ceph-mon[97920]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:01 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3401069633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 -- 172.21.15.66:0/2756733543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d90102040 msgr2=0x7f6d9010e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2756733543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d90102040 0x7f6d9010e430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b0a0 tx=0x7f6d8800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 -- 172.21.15.66:0/2756733543 shutdown_connections 2023-12-11T19:31:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2756733543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d901076f0 0x7f6d90107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2756733543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d90102040 0x7f6d9010e430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2756733543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 0x7f6d90101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 -- 172.21.15.66:0/2756733543 >> 172.21.15.66:0/2756733543 conn(0x7f6d900787c0 msgr2=0x7f6d90078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 -- 172.21.15.66:0/2756733543 shutdown_connections 2023-12-11T19:31:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.351+0000 7f6d95f09700 1 -- 172.21.15.66:0/2756733543 wait complete. 2023-12-11T19:31:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.353+0000 7f6d95f09700 1 Processor -- start 2023-12-11T19:31:03.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.353+0000 7f6d95f09700 1 -- start start 2023-12-11T19:31:03.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.353+0000 7f6d95f09700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 0x7f6d9019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:03.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d95f09700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d90102040 0x7f6d9019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:03.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d95f09700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 0x7f6d901978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:03.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d95f09700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90197f20 con 0x7f6d901076f0 2023-12-11T19:31:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d95f09700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90198090 con 0x7f6d90102040 2023-12-11T19:31:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d95f09700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90198250 con 0x7f6d90101680 2023-12-11T19:31:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 0x7f6d901978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 0x7f6d901978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41860/0 (socket says 172.21.15.66:41860) 2023-12-11T19:31:03.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 -- 172.21.15.66:0/2976188944 learned_addr learned my addr 172.21.15.66:0/2976188944 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8f7fe700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 0x7f6d9019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8effd700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d90102040 0x7f6d9019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 -- 172.21.15.66:0/2976188944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 msgr2=0x7f6d9019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 0x7f6d9019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 -- 172.21.15.66:0/2976188944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d90102040 msgr2=0x7f6d9019dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d90102040 0x7f6d9019dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8ffff700 1 -- 172.21.15.66:0/2976188944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009d00 con 0x7f6d901076f0 2023-12-11T19:31:03.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.354+0000 7f6d8f7fe700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 0x7f6d9019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.355+0000 7f6d8ffff700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 0x7f6d901978c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b070 tx=0x7f6d8800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.355+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800e040 con 0x7f6d901076f0 2023-12-11T19:31:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.355+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d88003c00 con 0x7f6d901076f0 2023-12-11T19:31:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.355+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d901984d0 con 0x7f6d901076f0 2023-12-11T19:31:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.355+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88005040 con 0x7f6d901076f0 2023-12-11T19:31:03.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.355+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d901a4770 con 0x7f6d901076f0 2023-12-11T19:31:03.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.357+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d90109b80 con 0x7f6d901076f0 2023-12-11T19:31:03.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.358+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6d8801d480 con 0x7f6d901076f0 2023-12-11T19:31:03.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.360+0000 7f6d8cff9700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6d780a1480 0x7f6d780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:03.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.360+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d880cdb00 con 0x7f6d901076f0 2023-12-11T19:31:03.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.360+0000 7f6d8f7fe700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6d780a1480 0x7f6d780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:03.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.361+0000 7f6d8f7fe700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6d780a1480 0x7f6d780a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009c90 tx=0x7f6d80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:03.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.364+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6d880cfe60 con 0x7f6d901076f0 2023-12-11T19:31:03.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.573+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d9004f2e0 con 0x7f6d901076f0 2023-12-11T19:31:03.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.573+0000 7f6d8cff9700 1 -- 172.21.15.66:0/2976188944 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d880975f0 con 0x7f6d901076f0 2023-12-11T19:31:03.575 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:31:03.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6d780a1480 msgr2=0x7f6d780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:03.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6d780a1480 0x7f6d780a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009c90 tx=0x7f6d80009260 comp rx=0 tx=0).stop 2023-12-11T19:31:03.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 msgr2=0x7f6d901978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:03.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 0x7f6d901978c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b070 tx=0x7f6d8800bed0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 shutdown_connections 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6d780a1480 0x7f6d780a3940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d901076f0 0x7f6d901978c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d90101680 0x7f6d9019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 --2- 172.21.15.66:0/2976188944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d90102040 0x7f6d9019dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.575+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 >> 172.21.15.66:0/2976188944 conn(0x7f6d900787c0 msgr2=0x7f6d90100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.576+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 shutdown_connections 2023-12-11T19:31:03.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:03.576+0000 7f6d95f09700 1 -- 172.21.15.66:0/2976188944 wait complete. 2023-12-11T19:31:03.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:03 smithi066 ceph-mon[97920]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:03.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:03 smithi167 ceph-mon[102750]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:03.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:03 smithi070 ceph-mon[104044]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.a on smithi066 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.d on smithi066 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.g on smithi066 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.j on smithi066 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.b on smithi070 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.e on smithi070 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.h on smithi070 2023-12-11T19:31:04.061 INFO:tasks.cephadm:Adding mds.k on smithi070 2023-12-11T19:31:04.062 INFO:tasks.cephadm:Adding mds.c on smithi167 2023-12-11T19:31:04.062 INFO:tasks.cephadm:Adding mds.f on smithi167 2023-12-11T19:31:04.062 INFO:tasks.cephadm:Adding mds.i on smithi167 2023-12-11T19:31:04.062 INFO:tasks.cephadm:Adding mds.l on smithi167 2023-12-11T19:31:04.062 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph orch apply mds all '12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2023-12-11T19:31:04.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:04 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2976188944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:04.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:04 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2976188944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:04.853 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:31:04.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:04 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2976188944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:05.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:05 smithi167 ceph-mon[102750]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:05.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:05 smithi066 ceph-mon[97920]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:05.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:05 smithi070 ceph-mon[104044]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.376+0000 7fb67aa40700 1 -- 172.21.15.167:0/1715774349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb674102820 msgr2=0x7fb674102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.376+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1715774349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb674102820 0x7fb674102c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb668009990 tx=0x7fb668009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 -- 172.21.15.167:0/1715774349 shutdown_connections 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1715774349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6741031c0 0x7fb674109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1715774349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 0x7fb674101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1715774349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb674102820 0x7fb674102c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 -- 172.21.15.167:0/1715774349 >> 172.21.15.167:0/1715774349 conn(0x7fb6740fcbe0 msgr2=0x7fb6740ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:06.379 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 -- 172.21.15.167:0/1715774349 shutdown_connections 2023-12-11T19:31:06.380 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.377+0000 7fb67aa40700 1 -- 172.21.15.167:0/1715774349 wait complete. 2023-12-11T19:31:06.380 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 Processor -- start 2023-12-11T19:31:06.380 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 -- start start 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 0x7fb674197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb674102820 0x7fb674197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 0x7fb67419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67419ef70 con 0x7fb674101620 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67419f0b0 con 0x7fb674102820 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb67aa40700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67419f240 con 0x7fb6741031c0 2023-12-11T19:31:06.381 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb678fdd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 0x7fb67419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb678fdd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 0x7fb67419e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59630/0 (socket says 172.21.15.167:59630) 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.378+0000 7fb678fdd700 1 -- 172.21.15.167:0/1027484701 learned_addr learned my addr 172.21.15.167:0/1027484701 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb673fff700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 0x7fb674197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb6737fe700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb674102820 0x7fb674197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb678fdd700 1 -- 172.21.15.167:0/1027484701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb674102820 msgr2=0x7fb674197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb678fdd700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb674102820 0x7fb674197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb678fdd700 1 -- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 msgr2=0x7fb674197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb678fdd700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 0x7fb674197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb678fdd700 1 -- 172.21.15.167:0/1027484701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6680095f0 con 0x7fb6741031c0 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb673fff700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 0x7fb674197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb678fdd700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 0x7fb67419e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb66c007f10 tx=0x7fb66c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00ddd0 con 0x7fb6741031c0 2023-12-11T19:31:06.382 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb66c00f040 con 0x7fb6741031c0 2023-12-11T19:31:06.383 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c004870 con 0x7fb6741031c0 2023-12-11T19:31:06.383 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.379+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67419f520 con 0x7fb6741031c0 2023-12-11T19:31:06.383 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.380+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67419f9d0 con 0x7fb6741031c0 2023-12-11T19:31:06.383 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.381+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb674108b20 con 0x7fb6741031c0 2023-12-11T19:31:06.384 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.383+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb66c003a00 con 0x7fb6741031c0 2023-12-11T19:31:06.385 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.384+0000 7fb6717fa700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb65c0a1500 0x7fb65c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:06.385 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.384+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb66c0cc2c0 con 0x7fb6741031c0 2023-12-11T19:31:06.386 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.384+0000 7fb673fff700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb65c0a1500 0x7fb65c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:06.386 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.385+0000 7fb673fff700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb65c0a1500 0x7fb65c0a39c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb674102680 tx=0x7fb66400a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:06.389 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.388+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb66c04f2f0 con 0x7fb6741031c0 2023-12-11T19:31:06.601 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.599+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fb67402d480 con 0x7fb65c0a1500 2023-12-11T19:31:06.609 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.608+0000 7fb6717fa700 1 -- 172.21.15.167:0/1027484701 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb67402d480 con 0x7fb65c0a1500 2023-12-11T19:31:06.609 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb65c0a1500 msgr2=0x7fb65c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb65c0a1500 0x7fb65c0a39c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb674102680 tx=0x7fb66400a680 comp rx=0 tx=0).stop 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 msgr2=0x7fb67419e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 0x7fb67419e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb66c007f10 tx=0x7fb66c00d3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 shutdown_connections 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb65c0a1500 0x7fb65c0a39c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6741031c0 0x7fb67419e9a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb674101620 0x7fb674197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.612 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 --2- 172.21.15.167:0/1027484701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb674102820 0x7fb674197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:06.613 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.610+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 >> 172.21.15.167:0/1027484701 conn(0x7fb6740fcbe0 msgr2=0x7fb674105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:06.613 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.611+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 shutdown_connections 2023-12-11T19:31:06.613 INFO:teuthology.orchestra.run.smithi167.stderr:2023-12-11T19:31:06.611+0000 7fb67aa40700 1 -- 172.21.15.167:0/1027484701 wait complete. 2023-12-11T19:31:07.140 DEBUG:teuthology.orchestra.run.smithi066:mds.a> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.a.service 2023-12-11T19:31:07.146 DEBUG:teuthology.orchestra.run.smithi066:mds.d> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.d.service 2023-12-11T19:31:07.191 DEBUG:teuthology.orchestra.run.smithi066:mds.g> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.g.service 2023-12-11T19:31:07.229 INFO:journalctl@ceph.mds.a.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:31:07.233 DEBUG:teuthology.orchestra.run.smithi066:mds.j> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.j.service 2023-12-11T19:31:07.266 INFO:journalctl@ceph.mds.d.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:31:07.269 DEBUG:teuthology.orchestra.run.smithi070:mds.b> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.b.service 2023-12-11T19:31:07.275 DEBUG:teuthology.orchestra.run.smithi070:mds.e> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.e.service 2023-12-11T19:31:07.307 INFO:journalctl@ceph.mds.g.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:31:07.323 DEBUG:teuthology.orchestra.run.smithi070:mds.h> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.h.service 2023-12-11T19:31:07.341 INFO:journalctl@ceph.mds.j.smithi066.stdout:-- Logs begin at Mon 2023-12-11 19:05:32 UTC. -- 2023-12-11T19:31:07.348 INFO:journalctl@ceph.mds.b.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:31:07.353 DEBUG:teuthology.orchestra.run.smithi070:mds.k> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.k.service 2023-12-11T19:31:07.390 INFO:journalctl@ceph.mds.e.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:31:07.394 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.c.service 2023-12-11T19:31:07.398 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.f.service 2023-12-11T19:31:07.425 INFO:journalctl@ceph.mds.h.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:31:07.443 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.i.service 2023-12-11T19:31:07.462 INFO:journalctl@ceph.mds.k.smithi070.stdout:-- Logs begin at Mon 2023-12-11 19:04:26 UTC. -- 2023-12-11T19:31:07.471 INFO:journalctl@ceph.mds.c.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:31:07.474 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.l.service 2023-12-11T19:31:07.520 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-11T19:31:07.521 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:31:07.521 INFO:journalctl@ceph.mds.f.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:31:07.546 INFO:journalctl@ceph.mds.i.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:31:07.588 INFO:journalctl@ceph.mds.l.smithi167.stdout:-- Logs begin at Mon 2023-12-11 19:04:51 UTC. -- 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.748+0000 7f03b0bff700 1 -- 172.21.15.66:0/1116909479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac103030 msgr2=0x7f03ac1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.748+0000 7f03b0bff700 1 --2- 172.21.15.66:0/1116909479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac103030 0x7f03ac1098e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b0d0 tx=0x7f03a400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 -- 172.21.15.66:0/1116909479 shutdown_connections 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 --2- 172.21.15.66:0/1116909479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac103030 0x7f03ac1098e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 --2- 172.21.15.66:0/1116909479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ac1014a0 0x7f03ac1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 --2- 172.21.15.66:0/1116909479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac102690 0x7f03ac102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 -- 172.21.15.66:0/1116909479 >> 172.21.15.66:0/1116909479 conn(0x7f03ac0fca10 msgr2=0x7f03ac0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:07.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 -- 172.21.15.66:0/1116909479 shutdown_connections 2023-12-11T19:31:07.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.749+0000 7f03b0bff700 1 -- 172.21.15.66:0/1116909479 wait complete. 2023-12-11T19:31:07.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 Processor -- start 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 -- start start 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 0x7f03ac199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ac102690 0x7f03ac199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac103030 0x7f03ac1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1a1160 con 0x7f03ac1014a0 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1a12a0 con 0x7f03ac103030 2023-12-11T19:31:07.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.750+0000 7f03b0bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1a1430 con 0x7f03ac102690 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 0x7f03ac199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 0x7f03ac199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41878/0 (socket says 172.21.15.66:41878) 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 -- 172.21.15.66:0/2595985113 learned_addr learned my addr 172.21.15.66:0/2595985113 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03a3fff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ac102690 0x7f03ac199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aad9d700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac103030 0x7f03ac1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 -- 172.21.15.66:0/2595985113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ac102690 msgr2=0x7f03ac199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ac102690 0x7f03ac199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 -- 172.21.15.66:0/2595985113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac103030 msgr2=0x7f03ac1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac103030 0x7f03ac1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 -- 172.21.15.66:0/2595985113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394009520 con 0x7f03ac1014a0 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03aa59c700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 0x7f03ac199630 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e970 tx=0x7f039c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c009790 con 0x7f03ac1014a0 2023-12-11T19:31:07.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.751+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f039c00c7e0 con 0x7f03ac1014a0 2023-12-11T19:31:07.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.752+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a4009d00 con 0x7f03ac1014a0 2023-12-11T19:31:07.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.752+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c003bd0 con 0x7f03ac1014a0 2023-12-11T19:31:07.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.752+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac1a1ae0 con 0x7f03ac1014a0 2023-12-11T19:31:07.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.753+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac068230 con 0x7f03ac1014a0 2023-12-11T19:31:07.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.755+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f039c00c950 con 0x7f03ac1014a0 2023-12-11T19:31:07.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.756+0000 7f03a37fe700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f03980a1500 0x7f03980a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:07.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.756+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f039c014070 con 0x7f03ac1014a0 2023-12-11T19:31:07.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.756+0000 7f03a3fff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f03980a1500 0x7f03980a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:07.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.757+0000 7f03a3fff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f03980a1500 0x7f03980a39c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0394009df0 tx=0x7f0394009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.760+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f039c0964c0 con 0x7f03ac1014a0 2023-12-11T19:31:07.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:07.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:07.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:07.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:07.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:07.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.973+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03ac10cdb0 con 0x7f03ac1014a0 2023-12-11T19:31:07.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.974+0000 7f03a37fe700 1 -- 172.21.15.66:0/2595985113 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18314 (secure 0 0 0) 0x7f039c099ae0 con 0x7f03ac1014a0 2023-12-11T19:31:07.976 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:07.976 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:30:58.331837+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:31:07.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.976+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f03980a1500 msgr2=0x7f03980a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:07.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.976+0000 7f03b0bff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f03980a1500 0x7f03980a39c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0394009df0 tx=0x7f0394009260 comp rx=0 tx=0).stop 2023-12-11T19:31:07.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 msgr2=0x7f03ac199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:07.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 0x7f03ac199630 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e970 tx=0x7f039c00ec80 comp rx=0 tx=0).stop 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 shutdown_connections 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f03980a1500 0x7f03980a39c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac103030 0x7f03ac1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03ac1014a0 0x7f03ac199630 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 --2- 172.21.15.66:0/2595985113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03ac102690 0x7f03ac199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 >> 172.21.15.66:0/2595985113 conn(0x7f03ac0fca10 msgr2=0x7f03ac073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 shutdown_connections 2023-12-11T19:31:07.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:07.977+0000 7f03b0bff700 1 -- 172.21.15.66:0/2595985113 wait complete. 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:07.997 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:07.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:31:07.999 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.207+0000 7fa6279ff700 1 -- 172.21.15.66:0/1463755717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620104930 msgr2=0x7fa620106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.207+0000 7fa6279ff700 1 --2- 172.21.15.66:0/1463755717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620104930 0x7fa620106d20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6100098f0 tx=0x7fa610009c00 comp rx=0 tx=0).stop 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 -- 172.21.15.66:0/1463755717 shutdown_connections 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 --2- 172.21.15.66:0/1463755717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa620107260 0x7fa6201096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 --2- 172.21.15.66:0/1463755717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 0x7fa6201043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 --2- 172.21.15.66:0/1463755717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620104930 0x7fa620106d20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 -- 172.21.15.66:0/1463755717 >> 172.21.15.66:0/1463755717 conn(0x7fa6200fb9f0 msgr2=0x7fa6200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 -- 172.21.15.66:0/1463755717 shutdown_connections 2023-12-11T19:31:08.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.208+0000 7fa6279ff700 1 -- 172.21.15.66:0/1463755717 wait complete. 2023-12-11T19:31:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 Processor -- start 2023-12-11T19:31:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 -- start start 2023-12-11T19:31:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 0x7fa620101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa620104930 0x7fa6200fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620107260 0x7fa6201004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa620100c90 con 0x7fa620107260 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa620100e00 con 0x7fa620104930 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa6279ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201a1520 con 0x7fa620102000 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa62579b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 0x7fa620101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.209+0000 7fa625f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620107260 0x7fa6201004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa62579b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 0x7fa620101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:36946/0 (socket says 172.21.15.66:36946) 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa62579b700 1 -- 172.21.15.66:0/2434425398 learned_addr learned my addr 172.21.15.66:0/2434425398 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa624f9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa620104930 0x7fa6200fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa625f9c700 1 -- 172.21.15.66:0/2434425398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 msgr2=0x7fa620101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa625f9c700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 0x7fa620101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa625f9c700 1 -- 172.21.15.66:0/2434425398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa620104930 msgr2=0x7fa6200fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa625f9c700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa620104930 0x7fa6200fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa625f9c700 1 -- 172.21.15.66:0/2434425398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610009520 con 0x7fa620107260 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa625f9c700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620107260 0x7fa6201004f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa61800efc0 tx=0x7fa61800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618018070 con 0x7fa620107260 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61800f040 con 0x7fa620107260 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6201a1750 con 0x7fa620107260 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800cb40 con 0x7fa620107260 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.210+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6201a1c60 con 0x7fa620107260 2023-12-11T19:31:08.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.211+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6200fd570 con 0x7fa620107260 2023-12-11T19:31:08.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.214+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa618003a00 con 0x7fa620107260 2023-12-11T19:31:08.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.215+0000 7fa6167fc700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa60c0a1500 0x7fa60c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.215+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6180ccc30 con 0x7fa620107260 2023-12-11T19:31:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.215+0000 7fa62579b700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa60c0a1500 0x7fa60c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.216+0000 7fa62579b700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa60c0a1500 0x7fa60c0a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005950 tx=0x7fa61c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:08.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.217+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa6180967c0 con 0x7fa620107260 2023-12-11T19:31:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.430+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa62010cee0 con 0x7fa620107260 2023-12-11T19:31:08.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.431+0000 7fa6167fc700 1 -- 172.21.15.66:0/2434425398 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa618099de0 con 0x7fa620107260 2023-12-11T19:31:08.432 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa60c0a1500 msgr2=0x7fa60c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa60c0a1500 0x7fa60c0a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005950 tx=0x7fa61c009310 comp rx=0 tx=0).stop 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620107260 msgr2=0x7fa6201004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620107260 0x7fa6201004f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa61800efc0 tx=0x7fa61800c3a0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 shutdown_connections 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa60c0a1500 0x7fa60c0a39c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa620107260 0x7fa6201004f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa620102000 0x7fa620101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 --2- 172.21.15.66:0/2434425398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa620104930 0x7fa6200fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 >> 172.21.15.66:0/2434425398 conn(0x7fa6200fb9f0 msgr2=0x7fa620102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 shutdown_connections 2023-12-11T19:31:08.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.433+0000 7fa6279ff700 1 -- 172.21.15.66:0/2434425398 wait complete. 2023-12-11T19:31:08.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 -- 172.21.15.66:0/1465506195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c103030 msgr2=0x7f284c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 --2- 172.21.15.66:0/1465506195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c103030 0x7f284c1098e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f284400b0d0 tx=0x7f284400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 -- 172.21.15.66:0/1465506195 shutdown_connections 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 --2- 172.21.15.66:0/1465506195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f284c1014a0 0x7f284c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 --2- 172.21.15.66:0/1465506195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c103030 0x7f284c1098e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 --2- 172.21.15.66:0/1465506195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c102690 0x7f284c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 -- 172.21.15.66:0/1465506195 >> 172.21.15.66:0/1465506195 conn(0x7f284c0fca10 msgr2=0x7f284c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 -- 172.21.15.66:0/1465506195 shutdown_connections 2023-12-11T19:31:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.664+0000 7f2852dff700 1 -- 172.21.15.66:0/1465506195 wait complete. 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.665+0000 7f2852dff700 1 Processor -- start 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.665+0000 7f2852dff700 1 -- start start 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2852dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c1014a0 0x7f284c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2852dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 0x7f284c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2852dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f284c103030 0x7f284c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2852dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c1a1160 con 0x7f284c102690 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2852dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c1a12a0 con 0x7f284c103030 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2852dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c1a1430 con 0x7f284c1014a0 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 0x7f284c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2850b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c1014a0 0x7f284c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 0x7f284c199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41924/0 (socket says 172.21.15.66:41924) 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 -- 172.21.15.66:0/692065367 learned_addr learned my addr 172.21.15.66:0/692065367 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f285139c700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f284c103030 0x7f284c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 -- 172.21.15.66:0/692065367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c1014a0 msgr2=0x7f284c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c1014a0 0x7f284c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 -- 172.21.15.66:0/692065367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f284c103030 msgr2=0x7f284c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f284c103030 0x7f284c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f284bfff700 1 -- 172.21.15.66:0/692065367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844009d00 con 0x7f284c102690 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.666+0000 7f2850b9b700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c1014a0 0x7f284c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.667+0000 7f284bfff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 0x7f284c199c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f283800e9a0 tx=0x7f283800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.667+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838009790 con 0x7f284c102690 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.667+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f283800cbd0 con 0x7f284c102690 2023-12-11T19:31:08.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.667+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c1a1710 con 0x7f284c102690 2023-12-11T19:31:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.667+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838018680 con 0x7f284c102690 2023-12-11T19:31:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.667+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c1a1c20 con 0x7f284c102690 2023-12-11T19:31:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.668+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c068230 con 0x7f284c102690 2023-12-11T19:31:08.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.670+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f283800cd40 con 0x7f284c102690 2023-12-11T19:31:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.673+0000 7f2849ffb700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f283c0a1480 0x7f283c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.673+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2838014070 con 0x7f284c102690 2023-12-11T19:31:08.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.673+0000 7f2850b9b700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f283c0a1480 0x7f283c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:08.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.673+0000 7f2850b9b700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f283c0a1480 0x7f283c0a3940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f284c1024f0 tx=0x7f2840009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:08.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.675+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f28380522f0 con 0x7f284c102690 2023-12-11T19:31:08.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:08 smithi066 ceph-mon[97920]: Deploying daemon mds.l on smithi167 2023-12-11T19:31:08.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:08 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2595985113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:08.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:08 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2434425398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:08.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:08 smithi167 ceph-mon[102750]: Deploying daemon mds.l on smithi167 2023-12-11T19:31:08.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:08 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2595985113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:08.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:08 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2434425398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:08.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.934+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f284c10cb70 con 0x7f284c102690 2023-12-11T19:31:08.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.935+0000 7f2849ffb700 1 -- 172.21.15.66:0/692065367 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f28380965e0 con 0x7f284c102690 2023-12-11T19:31:08.936 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:08.937 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-11T19:31:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f283c0a1480 msgr2=0x7f283c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f283c0a1480 0x7f283c0a3940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f284c1024f0 tx=0x7f2840009260 comp rx=0 tx=0).stop 2023-12-11T19:31:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 msgr2=0x7f284c199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 0x7f284c199c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f283800e9a0 tx=0x7f283800ecb0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 shutdown_connections 2023-12-11T19:31:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f283c0a1480 0x7f283c0a3940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f284c103030 0x7f284c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f284c1014a0 0x7f284c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 --2- 172.21.15.66:0/692065367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f284c102690 0x7f284c199c10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.937+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 >> 172.21.15.66:0/692065367 conn(0x7f284c0fca10 msgr2=0x7f284c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.938+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 shutdown_connections 2023-12-11T19:31:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:08.938+0000 7f2852dff700 1 -- 172.21.15.66:0/692065367 wait complete. 2023-12-11T19:31:08.941 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 1 2023-12-11T19:31:09.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:08 smithi070 ceph-mon[104044]: Deploying daemon mds.l on smithi167 2023-12-11T19:31:09.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:08 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2595985113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:09.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:08 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2434425398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:09.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:09 smithi167 ceph-mon[102750]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:09.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:09 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/692065367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:09.859 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:31:09 smithi167 systemd[1]: Starting Ceph mds.l for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:10.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:09 smithi066 ceph-mon[97920]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:10.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:09 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/692065367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:10.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:09 smithi070 ceph-mon[104044]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:10.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/692065367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:10.654 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:31:10 smithi167 podman[138982]: 2023-12-11T19:31:10.959 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:11.020 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:31:10 smithi167 bash[138982]: 393d0681aebd24a46d28e5d16c5ead7baf8e95300f83657acd68d72d3ef34a5f 2023-12-11T19:31:11.020 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:31:10 smithi167 ceph-mds[139022]: -- 172.21.15.167:0/905272757 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561cbca1d520 con 0x561cbc9f4800 2023-12-11T19:31:11.020 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:31:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-l[138998]: starting mds.l at 2023-12-11T19:31:11.021 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:31:10 smithi167 systemd[1]: Started Ceph mds.l for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:11.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.179+0000 7f8ab59ff700 1 -- 172.21.15.66:0/3613022126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0101430 msgr2=0x7f8ab0101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:11.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.179+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/3613022126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0101430 0x7f8ab0101850 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00099c0 tx=0x7f8aa0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 -- 172.21.15.66:0/3613022126 shutdown_connections 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/3613022126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0101430 0x7f8ab0101850 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/3613022126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/3613022126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab0102620 0x7f8ab0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 -- 172.21.15.66:0/3613022126 >> 172.21.15.66:0/3613022126 conn(0x7f8ab00fc980 msgr2=0x7f8ab00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 -- 172.21.15.66:0/3613022126 shutdown_connections 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.180+0000 7f8ab59ff700 1 -- 172.21.15.66:0/3613022126 wait complete. 2023-12-11T19:31:11.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.181+0000 7f8ab59ff700 1 Processor -- start 2023-12-11T19:31:11.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.181+0000 7f8ab59ff700 1 -- start start 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8ab59ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab0101430 0x7f8ab0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8ab59ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 0x7f8ab00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8ab59ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8ab59ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00746a0 con 0x7f8ab0102fc0 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8ab59ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab0074810 con 0x7f8ab0102620 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8ab59ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01a1370 con 0x7f8ab0101430 2023-12-11T19:31:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8aaf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8aae7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 0x7f8ab00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8aae7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 0x7f8ab00739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41440/0 (socket says 172.21.15.66:41440) 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8aaf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41940/0 (socket says 172.21.15.66:41940) 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8aae7fc700 1 -- 172.21.15.66:0/1259386401 learned_addr learned my addr 172.21.15.66:0/1259386401 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.182+0000 7f8aaeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab0101430 0x7f8ab0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aae7fc700 1 -- 172.21.15.66:0/1259386401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab0101430 msgr2=0x7f8ab0075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aae7fc700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab0101430 0x7f8ab0075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aae7fc700 1 -- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 msgr2=0x7f8ab0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aae7fc700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aae7fc700 1 -- 172.21.15.66:0/1259386401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00095f0 con 0x7f8ab0102620 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aaf7fe700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8aae7fc700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 0x7f8ab00739c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e9a0 tx=0x7f8aa400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4009790 con 0x7f8ab0102620 2023-12-11T19:31:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8aa400ce40 con 0x7f8ab0102620 2023-12-11T19:31:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab01a1600 con 0x7f8ab0102620 2023-12-11T19:31:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4018420 con 0x7f8ab0102620 2023-12-11T19:31:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.183+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab01a1b10 con 0x7f8ab0102620 2023-12-11T19:31:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.184+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab0108970 con 0x7f8ab0102620 2023-12-11T19:31:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.186+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8aa4010430 con 0x7f8ab0102620 2023-12-11T19:31:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.188+0000 7f8ab49fd700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8a980a1480 0x7f8a980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.188+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8aa4014070 con 0x7f8ab0102620 2023-12-11T19:31:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.188+0000 7f8aaeffd700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8a980a1480 0x7f8a980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.189+0000 7f8aaeffd700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8a980a1480 0x7f8a980a3940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0000c00 tx=0x7f8aa0003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.192+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8aa40522f0 con 0x7f8ab0102620 2023-12-11T19:31:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.452+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ab00681c0 con 0x7f8ab0102620 2023-12-11T19:31:11.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.453+0000 7f8ab49fd700 1 -- 172.21.15.66:0/1259386401 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f8aa4096510 con 0x7f8ab0102620 2023-12-11T19:31:11.453 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:11.453 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-11T19:31:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8a980a1480 msgr2=0x7f8a980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8a980a1480 0x7f8a980a3940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0000c00 tx=0x7f8aa0003680 comp rx=0 tx=0).stop 2023-12-11T19:31:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 msgr2=0x7f8ab00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 0x7f8ab00739c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e9a0 tx=0x7f8aa400ecb0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 shutdown_connections 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f8a980a1480 0x7f8a980a3940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ab0102fc0 0x7f8ab0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab0101430 0x7f8ab0075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 --2- 172.21.15.66:0/1259386401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ab0102620 0x7f8ab00739c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 >> 172.21.15.66:0/1259386401 conn(0x7f8ab00fc980 msgr2=0x7f8ab0105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 shutdown_connections 2023-12-11T19:31:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:11.455+0000 7f8ab59ff700 1 -- 172.21.15.66:0/1259386401 wait complete. 2023-12-11T19:31:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.167:6832/1537942709,v1:172.21.15.167:6833/1537942709] up:boot 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: fsmap 1 up:standby 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:11.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:11.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:11.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:11.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:11.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: Deploying daemon mds.k on smithi070 2023-12-11T19:31:11.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:11 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1259386401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:11.990 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:11.991 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.167:6832/1537942709,v1:172.21.15.167:6833/1537942709] up:boot 2023-12-11T19:31:11.991 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: fsmap 1 up:standby 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: Deploying daemon mds.k on smithi070 2023-12-11T19:31:11.992 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:11 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1259386401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.167:6832/1537942709,v1:172.21.15.167:6833/1537942709] up:boot 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: fsmap 1 up:standby 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: Deploying daemon mds.k on smithi070 2023-12-11T19:31:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:11 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1259386401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:13.122 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:31:13 smithi070 systemd[1]: Starting Ceph mds.k for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:13.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 -- 172.21.15.66:0/3209699374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 msgr2=0x7f5818102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 --2- 172.21.15.66:0/3209699374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818102af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f580c0099c0 tx=0x7f580c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 -- 172.21.15.66:0/3209699374 shutdown_connections 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 --2- 172.21.15.66:0/3209699374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5818103030 0x7f58181098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 --2- 172.21.15.66:0/3209699374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 0x7f58181018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 --2- 172.21.15.66:0/3209699374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818102af0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 -- 172.21.15.66:0/3209699374 >> 172.21.15.66:0/3209699374 conn(0x7f58180fca10 msgr2=0x7f58180fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:13.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 -- 172.21.15.66:0/3209699374 shutdown_connections 2023-12-11T19:31:13.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.696+0000 7f581e3ff700 1 -- 172.21.15.66:0/3209699374 wait complete. 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.697+0000 7f581e3ff700 1 Processor -- start 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.697+0000 7f581e3ff700 1 -- start start 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.697+0000 7f581e3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 0x7f5818199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.697+0000 7f581e3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f581e3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5818103030 0x7f58181a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f581e3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58181a1090 con 0x7f5818102690 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f581e3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58181a11d0 con 0x7f5818103030 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f581e3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58181a1360 con 0x7f58181014a0 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39166/0 (socket says 172.21.15.66:39166) 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 -- 172.21.15.66:0/181276039 learned_addr learned my addr 172.21.15.66:0/181276039 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:13.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f581c99c700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5818103030 0x7f58181a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f5817fff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 0x7f5818199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 -- 172.21.15.66:0/181276039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 msgr2=0x7f5818199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 0x7f5818199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 -- 172.21.15.66:0/181276039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5818103030 msgr2=0x7f58181a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5818103030 0x7f58181a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 -- 172.21.15.66:0/181276039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0095f0 con 0x7f5818102690 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f5817fff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 0x7f5818199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.698+0000 7f58177fe700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818199aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f580c009fd0 tx=0x7f580c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.699+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c01d070 con 0x7f5818102690 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.699+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58181a15e0 con 0x7f5818102690 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.699+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580c00bd10 con 0x7f5818102690 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.699+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c004bb0 con 0x7f5818102690 2023-12-11T19:31:13.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.699+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58181a1a10 con 0x7f5818102690 2023-12-11T19:31:13.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.700+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5818068230 con 0x7f5818102690 2023-12-11T19:31:13.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.702+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f580c004d10 con 0x7f5818102690 2023-12-11T19:31:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.703+0000 7f58157fa700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f58000a1480 0x7f58000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.703+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f580c0cd730 con 0x7f5818102690 2023-12-11T19:31:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.703+0000 7f5817fff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f58000a1480 0x7f58000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:13.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.704+0000 7f5817fff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f58000a1480 0x7f58000a3940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58181024f0 tx=0x7f580800b670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:13.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.708+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f580c097220 con 0x7f5818102690 2023-12-11T19:31:13.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:13 smithi066 ceph-mon[97920]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:13.882 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:13 smithi070 ceph-mon[104044]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:13.882 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:31:13 smithi070 podman[139143]: 2023-12-11T19:31:13.882 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:31:13 smithi070 bash[139143]: c6679829221b7869207d302e4f1c92e17fd2c36cad2791d9c63220b66b902b8e 2023-12-11T19:31:13.882 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:31:13 smithi070 ceph-mds[139183]: -- 172.21.15.70:0/3531291568 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ada59b520 con 0x559ada572800 2023-12-11T19:31:13.883 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:31:13 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-k[139159]: starting mds.k at 2023-12-11T19:31:13.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.971+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f581810cbc0 con 0x7f5818102690 2023-12-11T19:31:13.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.971+0000 7f58157fa700 1 -- 172.21.15.66:0/181276039 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f580c09a840 con 0x7f5818102690 2023-12-11T19:31:13.972 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:13.973 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-11T19:31:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f58000a1480 msgr2=0x7f58000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f58000a1480 0x7f58000a3940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58181024f0 tx=0x7f580800b670 comp rx=0 tx=0).stop 2023-12-11T19:31:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 msgr2=0x7f5818199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818199aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f580c009fd0 tx=0x7f580c00fa00 comp rx=0 tx=0).stop 2023-12-11T19:31:13.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 shutdown_connections 2023-12-11T19:31:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f58000a1480 0x7f58000a3940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5818103030 0x7f58181a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58181014a0 0x7f5818199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 --2- 172.21.15.66:0/181276039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5818102690 0x7f5818199aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 >> 172.21.15.66:0/181276039 conn(0x7f58180fca10 msgr2=0x7f5818107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:13.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 shutdown_connections 2023-12-11T19:31:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:13.974+0000 7f581e3ff700 1 -- 172.21.15.66:0/181276039 wait complete. 2023-12-11T19:31:13.978 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2023-12-11T19:31:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:13 smithi167 ceph-mon[102750]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:14.360 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:31:13 smithi070 systemd[1]: Started Ceph mds.k for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/181276039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: Deploying daemon mds.j on smithi066 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:boot 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: fsmap 2 up:standby 2023-12-11T19:31:15.248 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:14 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T19:31:15.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/181276039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: Deploying daemon mds.j on smithi066 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:boot 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: fsmap 2 up:standby 2023-12-11T19:31:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:14 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T19:31:15.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/181276039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:15.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: Deploying daemon mds.j on smithi066 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:boot 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: fsmap 2 up:standby 2023-12-11T19:31:15.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:14 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T19:31:15.995 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:16.225 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:16 smithi066 ceph-mon[97920]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:16.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:16 smithi167 ceph-mon[102750]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:16.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:16 smithi070 ceph-mon[104044]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 -- 172.21.15.66:0/1041764746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701048a0 msgr2=0x7fb670104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 --2- 172.21.15.66:0/1041764746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701048a0 0x7fb670104d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb6640099c0 tx=0x7fb664009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 -- 172.21.15.66:0/1041764746 shutdown_connections 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 --2- 172.21.15.66:0/1041764746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb670105240 0x7fb67010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 --2- 172.21.15.66:0/1041764746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6701036b0 0x7fb670103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 --2- 172.21.15.66:0/1041764746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701048a0 0x7fb670104d00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 -- 172.21.15.66:0/1041764746 >> 172.21.15.66:0/1041764746 conn(0x7fb6700fec40 msgr2=0x7fb670101060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 -- 172.21.15.66:0/1041764746 shutdown_connections 2023-12-11T19:31:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.382+0000 7fb6759ff700 1 -- 172.21.15.66:0/1041764746 wait complete. 2023-12-11T19:31:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.383+0000 7fb6759ff700 1 Processor -- start 2023-12-11T19:31:16.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.383+0000 7fb6759ff700 1 -- start start 2023-12-11T19:31:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb6759ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 0x7fb670199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb6759ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6701048a0 0x7fb6701999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb6759ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb670105240 0x7fb6701a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:16.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb6759ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701a0fb0 con 0x7fb6701036b0 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb6759ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701a10f0 con 0x7fb6701048a0 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb6759ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701a1280 con 0x7fb670105240 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 0x7fb670199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 0x7fb670199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39186/0 (socket says 172.21.15.66:39186) 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 -- 172.21.15.66:0/2713387898 learned_addr learned my addr 172.21.15.66:0/2713387898 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66e7fc700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6701048a0 0x7fb6701999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66f7fe700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb670105240 0x7fb6701a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 -- 172.21.15.66:0/2713387898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb670105240 msgr2=0x7fb6701a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb670105240 0x7fb6701a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 -- 172.21.15.66:0/2713387898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6701048a0 msgr2=0x7fb6701999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6701048a0 0x7fb6701999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66effd700 1 -- 172.21.15.66:0/2713387898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6640095f0 con 0x7fb6701036b0 2023-12-11T19:31:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.384+0000 7fb66f7fe700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb670105240 0x7fb6701a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.385+0000 7fb66effd700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 0x7fb670199480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c990 tx=0x7fb66000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.385+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660007b00 con 0x7fb6701036b0 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.385+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb660007c60 con 0x7fb6701036b0 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.385+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6701a1560 con 0x7fb6701036b0 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.385+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660018660 con 0x7fb6701036b0 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.385+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6701a1ac0 con 0x7fb6701036b0 2023-12-11T19:31:16.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.386+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67010aa60 con 0x7fb6701036b0 2023-12-11T19:31:16.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.390+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb66000f450 con 0x7fb6701036b0 2023-12-11T19:31:16.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.391+0000 7fb6749fd700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb6580a1500 0x7fb6580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:16.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.391+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6600cc870 con 0x7fb6701036b0 2023-12-11T19:31:16.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.392+0000 7fb66e7fc700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb6580a1500 0x7fb6580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:16.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.392+0000 7fb66e7fc700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb6580a1500 0x7fb6580a39c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c010 tx=0x7fb664005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:16.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.396+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb6600962d0 con 0x7fb6701036b0 2023-12-11T19:31:16.563 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:31:16 smithi066 systemd[1]: Starting Ceph mds.j for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:16.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.675+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb670068230 con 0x7fb6701036b0 2023-12-11T19:31:16.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.676+0000 7fb6749fd700 1 -- 172.21.15.66:0/2713387898 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fb6600998f0 con 0x7fb6701036b0 2023-12-11T19:31:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:16.677 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-11T19:31:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb6580a1500 msgr2=0x7fb6580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb6580a1500 0x7fb6580a39c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c010 tx=0x7fb664005fb0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 msgr2=0x7fb670199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:16.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 0x7fb670199480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c990 tx=0x7fb66000cd50 comp rx=0 tx=0).stop 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 shutdown_connections 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb6580a1500 0x7fb6580a39c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb670105240 0x7fb6701a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6701036b0 0x7fb670199480 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 --2- 172.21.15.66:0/2713387898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6701048a0 0x7fb6701999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.678+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 >> 172.21.15.66:0/2713387898 conn(0x7fb6700fec40 msgr2=0x7fb670107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.679+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 shutdown_connections 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:16.679+0000 7fb6759ff700 1 -- 172.21.15.66:0/2713387898 wait complete. 2023-12-11T19:31:16.680 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2023-12-11T19:31:16.970 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:31:16 smithi066 podman[143773]: 2023-12-11T19:31:17.312 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:31:16 smithi066 bash[143773]: 7c65d7fccb3d27380c5443e69b06b6f4f1a21970652a54aa6ac2c32db01e5a6b 2023-12-11T19:31:17.313 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:31:16 smithi066 ceph-mds[143815]: -- 172.21.15.66:0/775009592 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ba93c67520 con 0x55ba93c3e800 2023-12-11T19:31:17.313 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:31:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-j[143791]: starting mds.j at 2023-12-11T19:31:17.313 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:31:17 smithi066 systemd[1]: Started Ceph mds.j for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2713387898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:17 smithi167 ceph-mon[102750]: Deploying daemon mds.d on smithi066 2023-12-11T19:31:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2713387898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:17.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:17.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:17.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:17.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:17 smithi066 ceph-mon[97920]: Deploying daemon mds.d on smithi066 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2713387898' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:17.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:17 smithi070 ceph-mon[104044]: Deploying daemon mds.d on smithi066 2023-12-11T19:31:18.702 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:18.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:18 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.66:6834/1528570111,v1:172.21.15.66:6835/1528570111] up:boot 2023-12-11T19:31:18.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:18 smithi066 ceph-mon[97920]: fsmap 3 up:standby 2023-12-11T19:31:18.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:18 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T19:31:18.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:18 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.66:6834/1528570111,v1:172.21.15.66:6835/1528570111] up:boot 2023-12-11T19:31:18.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:18 smithi167 ceph-mon[102750]: fsmap 3 up:standby 2023-12-11T19:31:18.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:18 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T19:31:18.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:18 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.66:6834/1528570111,v1:172.21.15.66:6835/1528570111] up:boot 2023-12-11T19:31:18.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:18 smithi070 ceph-mon[104044]: fsmap 3 up:standby 2023-12-11T19:31:18.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:18 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 -- 172.21.15.66:0/2444273741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 msgr2=0x7fb7701098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 --2- 172.21.15.66:0/2444273741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb7701098e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb768009fc0 tx=0x7fb76800b330 comp rx=0 tx=0).stop 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 -- 172.21.15.66:0/2444273741 shutdown_connections 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 --2- 172.21.15.66:0/2444273741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7701014a0 0x7fb7701018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 --2- 172.21.15.66:0/2444273741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb7701098e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 --2- 172.21.15.66:0/2444273741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb770102690 0x7fb770102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:18.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 -- 172.21.15.66:0/2444273741 >> 172.21.15.66:0/2444273741 conn(0x7fb7700fca10 msgr2=0x7fb7700fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:18.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 -- 172.21.15.66:0/2444273741 shutdown_connections 2023-12-11T19:31:18.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.932+0000 7fb774fff700 1 -- 172.21.15.66:0/2444273741 wait complete. 2023-12-11T19:31:18.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.934+0000 7fb774fff700 1 Processor -- start 2023-12-11T19:31:18.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.934+0000 7fb774fff700 1 -- start start 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.934+0000 7fb774fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7701014a0 0x7fb770197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.934+0000 7fb774fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb770102690 0x7fb770197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.934+0000 7fb774fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb77019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb774fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77019ef60 con 0x7fb770103030 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb774fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77019f0a0 con 0x7fb7701014a0 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb774fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77019f230 con 0x7fb770102690 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb77019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7701014a0 0x7fb770197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:18.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb77019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39236/0 (socket says 172.21.15.66:39236) 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 -- 172.21.15.66:0/48725741 learned_addr learned my addr 172.21.15.66:0/48725741 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb767fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb770102690 0x7fb770197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 -- 172.21.15.66:0/48725741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb770102690 msgr2=0x7fb770197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb770102690 0x7fb770197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 -- 172.21.15.66:0/48725741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7701014a0 msgr2=0x7fb770197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7701014a0 0x7fb770197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.935+0000 7fb76ed9d700 1 -- 172.21.15.66:0/48725741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7600095f0 con 0x7fb770103030 2023-12-11T19:31:18.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.936+0000 7fb76ed9d700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb77019e990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb768008c80 tx=0x7fb768004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:18.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.936+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800e040 con 0x7fb770103030 2023-12-11T19:31:18.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.936+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb768008ee0 con 0x7fb770103030 2023-12-11T19:31:18.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.936+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768005040 con 0x7fb770103030 2023-12-11T19:31:18.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.936+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb768009c40 con 0x7fb770103030 2023-12-11T19:31:18.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.936+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77019f750 con 0x7fb770103030 2023-12-11T19:31:18.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.937+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7701088c0 con 0x7fb770103030 2023-12-11T19:31:18.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.939+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb76801d470 con 0x7fb770103030 2023-12-11T19:31:18.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.941+0000 7fb7677fe700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb75c0a1480 0x7fb75c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:18.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.941+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7680cdca0 con 0x7fb770103030 2023-12-11T19:31:18.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.941+0000 7fb76e59c700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb75c0a1480 0x7fb75c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:18.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.942+0000 7fb76e59c700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb75c0a1480 0x7fb75c0a3940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb7701024f0 tx=0x7fb760009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:18.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:18.946+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb768097790 con 0x7fb770103030 2023-12-11T19:31:19.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.231+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb770068230 con 0x7fb770103030 2023-12-11T19:31:19.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.232+0000 7fb7677fe700 1 -- 172.21.15.66:0/48725741 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7fb76809adb0 con 0x7fb770103030 2023-12-11T19:31:19.233 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:19.233 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-11T19:31:19.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.234+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb75c0a1480 msgr2=0x7fb75c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:19.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb75c0a1480 0x7fb75c0a3940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb7701024f0 tx=0x7fb760009310 comp rx=0 tx=0).stop 2023-12-11T19:31:19.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 msgr2=0x7fb77019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb77019e990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb768008c80 tx=0x7fb768004f90 comp rx=0 tx=0).stop 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 shutdown_connections 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb75c0a1480 0x7fb75c0a3940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7701014a0 0x7fb770197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb770103030 0x7fb77019e990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 --2- 172.21.15.66:0/48725741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb770102690 0x7fb770197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 >> 172.21.15.66:0/48725741 conn(0x7fb7700fca10 msgr2=0x7fb770105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 shutdown_connections 2023-12-11T19:31:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:19.235+0000 7fb774fff700 1 -- 172.21.15.66:0/48725741 wait complete. 2023-12-11T19:31:19.237 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 4 2023-12-11T19:31:19.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:19 smithi167 ceph-mon[102750]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:19.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:19 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/48725741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:19.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:19 smithi066 ceph-mon[97920]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:19.872 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:19 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/48725741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:19.872 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:31:19 smithi066 systemd[1]: Starting Ceph mds.d for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:19.873 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:19 smithi070 ceph-mon[104044]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:19.873 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:19 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/48725741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:20.169 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:31:19 smithi066 podman[144160]: 2023-12-11T19:31:20.562 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:31:20 smithi066 bash[144160]: 21701851f8cda2e96af7f64ba84b66546827428c4ef63c601c5ccb014e2df2aa 2023-12-11T19:31:20.563 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:31:20 smithi066 ceph-mds[144200]: -- 172.21.15.66:0/1044205531 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b852dff520 con 0x55b852dd6c00 2023-12-11T19:31:20.563 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:31:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-d[144176]: starting mds.d at 2023-12-11T19:31:20.563 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:31:20 smithi066 systemd[1]: Started Ceph mds.d for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:21.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:21.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 -- 172.21.15.66:0/1375497557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce401014a0 msgr2=0x7fce401018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 --2- 172.21.15.66:0/1375497557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce401014a0 0x7fce401018c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0099c0 tx=0x7fce3c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 -- 172.21.15.66:0/1375497557 shutdown_connections 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 --2- 172.21.15.66:0/1375497557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40103030 0x7fce401098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 --2- 172.21.15.66:0/1375497557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce401014a0 0x7fce401018c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 --2- 172.21.15.66:0/1375497557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce40102690 0x7fce40102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.482+0000 7fce47fff700 1 -- 172.21.15.66:0/1375497557 >> 172.21.15.66:0/1375497557 conn(0x7fce400fca10 msgr2=0x7fce400fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.483+0000 7fce47fff700 1 -- 172.21.15.66:0/1375497557 shutdown_connections 2023-12-11T19:31:21.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.483+0000 7fce47fff700 1 -- 172.21.15.66:0/1375497557 wait complete. 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.483+0000 7fce47fff700 1 Processor -- start 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 -- start start 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce401014a0 0x7fce40075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 0x7fce400739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 0x7fce40073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce400746a0 con 0x7fce40103030 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce40074810 con 0x7fce40102690 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce47fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce401a1360 con 0x7fce401014a0 2023-12-11T19:31:21.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce4659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 0x7fce40073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:21.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce4559a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 0x7fce400739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:21.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce4659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 0x7fce40073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39276/0 (socket says 172.21.15.66:39276) 2023-12-11T19:31:21.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce4559a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 0x7fce400739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:35660/0 (socket says 172.21.15.66:35660) 2023-12-11T19:31:21.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce4659c700 1 -- 172.21.15.66:0/385286195 learned_addr learned my addr 172.21.15.66:0/385286195 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:21.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.484+0000 7fce45d9b700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce401014a0 0x7fce40075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:21.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4659c700 1 -- 172.21.15.66:0/385286195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce401014a0 msgr2=0x7fce40075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4659c700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce401014a0 0x7fce40075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4659c700 1 -- 172.21.15.66:0/385286195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 msgr2=0x7fce400739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4659c700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 0x7fce400739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4659c700 1 -- 172.21.15.66:0/385286195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38009c40 con 0x7fce40103030 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4559a700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 0x7fce400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce4659c700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 0x7fce40073f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c370 tx=0x7fce3800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38017070 con 0x7fce40103030 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce3800f040 con 0x7fce40103030 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c0095f0 con 0x7fce40103030 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800e050 con 0x7fce40103030 2023-12-11T19:31:21.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.485+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce401a18c0 con 0x7fce40103030 2023-12-11T19:31:21.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.487+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce40108940 con 0x7fce40103030 2023-12-11T19:31:21.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.488+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fce3800cbb0 con 0x7fce40103030 2023-12-11T19:31:21.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.490+0000 7fce36ffd700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fce2c0a1480 0x7fce2c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:21.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.490+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fce380cc2e0 con 0x7fce40103030 2023-12-11T19:31:21.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.490+0000 7fce45d9b700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fce2c0a1480 0x7fce2c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:21.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.491+0000 7fce45d9b700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fce2c0a1480 0x7fce2c0a3940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009990 tx=0x7fce3c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:21.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.494+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fce38095d40 con 0x7fce40103030 2023-12-11T19:31:21.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: Deploying daemon mds.e on smithi070 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.66:6836/4055829654,v1:172.21.15.66:6837/4055829654] up:boot 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: fsmap 4 up:standby 2023-12-11T19:31:21.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:21 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T19:31:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:21.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:21.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:21.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: Deploying daemon mds.e on smithi070 2023-12-11T19:31:21.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.66:6836/4055829654,v1:172.21.15.66:6837/4055829654] up:boot 2023-12-11T19:31:21.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: fsmap 4 up:standby 2023-12-11T19:31:21.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:21 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T19:31:21.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.754+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce40068230 con 0x7fce40103030 2023-12-11T19:31:21.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.754+0000 7fce36ffd700 1 -- 172.21.15.66:0/385286195 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7fce38099360 con 0x7fce40103030 2023-12-11T19:31:21.755 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:21.755 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fce2c0a1480 msgr2=0x7fce2c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fce2c0a1480 0x7fce2c0a3940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009990 tx=0x7fce3c01a040 comp rx=0 tx=0).stop 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 msgr2=0x7fce40073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 0x7fce40073f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c370 tx=0x7fce3800c680 comp rx=0 tx=0).stop 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 shutdown_connections 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fce2c0a1480 0x7fce2c0a3940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce40103030 0x7fce40073f00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce401014a0 0x7fce40075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 --2- 172.21.15.66:0/385286195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce40102690 0x7fce400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:21.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 >> 172.21.15.66:0/385286195 conn(0x7fce400fca10 msgr2=0x7fce40105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:21.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 shutdown_connections 2023-12-11T19:31:21.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:21.757+0000 7fce47fff700 1 -- 172.21.15.66:0/385286195 wait complete. 2023-12-11T19:31:21.759 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 5 2023-12-11T19:31:21.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:21.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: Deploying daemon mds.e on smithi070 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.66:6836/4055829654,v1:172.21.15.66:6837/4055829654] up:boot 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: fsmap 4 up:standby 2023-12-11T19:31:21.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:21 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T19:31:22.618 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:22 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/385286195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:22.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:22 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/385286195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:22.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:22 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/385286195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:23.102 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:31:22 smithi070 systemd[1]: Starting Ceph mds.e for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:23.360 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:31:23 smithi070 podman[139483]: 2023-12-11T19:31:23.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:23 smithi167 ceph-mon[102750]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:23.779 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:23.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:23 smithi066 ceph-mon[97920]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:23.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:23 smithi070 ceph-mon[104044]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:23.860 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:31:23 smithi070 bash[139483]: fb1c463d68fb30e1d677d4a21276811bd7229908b2d09a4bbb8614c4563d178e 2023-12-11T19:31:23.860 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:31:23 smithi070 ceph-mds[139523]: -- 172.21.15.70:0/869235086 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd92037520 con 0x55cd9200f000 2023-12-11T19:31:23.860 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:31:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-e[139499]: starting mds.e at 2023-12-11T19:31:23.860 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:31:23 smithi070 systemd[1]: Started Ceph mds.e for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 -- 172.21.15.66:0/771644167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 msgr2=0x7f4390106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 --2- 172.21.15.66:0/771644167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390106c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4380009990 tx=0x7f4380009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 -- 172.21.15.66:0/771644167 shutdown_connections 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 --2- 172.21.15.66:0/771644167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4390101f00 0x7f43901042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 --2- 172.21.15.66:0/771644167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4390107160 0x7f43901095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 --2- 172.21.15.66:0/771644167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390106c20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 -- 172.21.15.66:0/771644167 >> 172.21.15.66:0/771644167 conn(0x7f43900fb8f0 msgr2=0x7f43900fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:24.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 -- 172.21.15.66:0/771644167 shutdown_connections 2023-12-11T19:31:24.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.998+0000 7f43987ff700 1 -- 172.21.15.66:0/771644167 wait complete. 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:23.999+0000 7f43987ff700 1 Processor -- start 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 -- start start 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4390101f00 0x7f4390199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4390107160 0x7f43901a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43901a1070 con 0x7f4390104830 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43901a11b0 con 0x7f4390107160 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f43987ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43901a1340 con 0x7f4390101f00 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39174/0 (socket says 172.21.15.66:39174) 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 -- 172.21.15.66:0/3403841721 learned_addr learned my addr 172.21.15.66:0/3403841721 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4396d9c700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4390107160 0x7f43901a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:24.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f439659b700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4390101f00 0x7f4390199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 -- 172.21.15.66:0/3403841721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4390101f00 msgr2=0x7f4390199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4390101f00 0x7f4390199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 -- 172.21.15.66:0/3403841721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4390107160 msgr2=0x7f43901a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4390107160 0x7f43901a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.000+0000 7f4395d9a700 1 -- 172.21.15.66:0/3403841721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800095f0 con 0x7f4390104830 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.001+0000 7f4395d9a700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390199a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4380006010 tx=0x7f4380004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.001+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438001d070 con 0x7f4390104830 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.001+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f438000bdb0 con 0x7f4390104830 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.001+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43901a15c0 con 0x7f4390104830 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.001+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438000fda0 con 0x7f4390104830 2023-12-11T19:31:24.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.001+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43901a19e0 con 0x7f4390104830 2023-12-11T19:31:24.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.002+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4390068230 con 0x7f4390104830 2023-12-11T19:31:24.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.004+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f438000f460 con 0x7f4390104830 2023-12-11T19:31:24.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.006+0000 7f43877fe700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f437c0a1480 0x7f437c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:24.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.006+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43800cdf60 con 0x7f4390104830 2023-12-11T19:31:24.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.006+0000 7f439659b700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f437c0a1480 0x7f437c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:24.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.006+0000 7f439659b700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f437c0a1480 0x7f437c0a3940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f438c009c60 tx=0x7f438c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:24.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.007+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4380097af0 con 0x7f4390104830 2023-12-11T19:31:24.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.267+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f439010cbe0 con 0x7f4390104830 2023-12-11T19:31:24.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.267+0000 7f43877fe700 1 -- 172.21.15.66:0/3403841721 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f438009b110 con 0x7f4390104830 2023-12-11T19:31:24.268 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:24.268 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f437c0a1480 msgr2=0x7f437c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f437c0a1480 0x7f437c0a3940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f438c009c60 tx=0x7f438c009260 comp rx=0 tx=0).stop 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 msgr2=0x7f4390199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390199a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4380006010 tx=0x7f4380004730 comp rx=0 tx=0).stop 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 shutdown_connections 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f437c0a1480 0x7f437c0a3940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4390107160 0x7f43901a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4390101f00 0x7f4390199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 --2- 172.21.15.66:0/3403841721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4390104830 0x7f4390199a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 >> 172.21.15.66:0/3403841721 conn(0x7f43900fb8f0 msgr2=0x7f43900fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 shutdown_connections 2023-12-11T19:31:24.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:24.270+0000 7f43987ff700 1 -- 172.21.15.66:0/3403841721 wait complete. 2023-12-11T19:31:24.272 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 6 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] up:boot 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: fsmap 5 up:standby 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: Deploying daemon mds.f on smithi167 2023-12-11T19:31:24.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:24 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3403841721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:24.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] up:boot 2023-12-11T19:31:24.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: fsmap 5 up:standby 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: Deploying daemon mds.f on smithi167 2023-12-11T19:31:24.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:24 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3403841721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:24.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] up:boot 2023-12-11T19:31:24.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: fsmap 5 up:standby 2023-12-11T19:31:24.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T19:31:24.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:24.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:24.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:24.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:24.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: Deploying daemon mds.f on smithi167 2023-12-11T19:31:24.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:24 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3403841721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:25.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:25 smithi066 ceph-mon[97920]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:25.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:25 smithi167 ceph-mon[102750]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:25.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:25 smithi070 ceph-mon[104044]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:26.270 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:31:25 smithi167 systemd[1]: Starting Ceph mds.f for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:26.292 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/2624651045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68102690 msgr2=0x7f9b68102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/2624651045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68102690 0x7f9b68102af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b580099c0 tx=0x7f9b58009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/2624651045 shutdown_connections 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/2624651045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68103030 0x7f9b681098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/2624651045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b681018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/2624651045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68102690 0x7f9b68102af0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/2624651045 >> 172.21.15.66:0/2624651045 conn(0x7f9b680fca10 msgr2=0x7f9b680fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/2624651045 shutdown_connections 2023-12-11T19:31:26.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.510+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/2624651045 wait complete. 2023-12-11T19:31:26.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.511+0000 7f9b6f5ff700 1 Processor -- start 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.511+0000 7f9b6f5ff700 1 -- start start 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6f5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b68075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6f5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68102690 0x7f9b680739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6f5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68103030 0x7f9b68073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6f5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680746a0 con 0x7f9b68103030 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6f5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b68074810 con 0x7f9b68102690 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6f5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b681a13a0 con 0x7f9b681014a0 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b68075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6db9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68103030 0x7f9b68073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b68075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37894/0 (socket says 172.21.15.66:37894) 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 -- 172.21.15.66:0/1473694297 learned_addr learned my addr 172.21.15.66:0/1473694297 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6cb9a700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68102690 0x7f9b680739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 -- 172.21.15.66:0/1473694297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68102690 msgr2=0x7f9b680739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68102690 0x7f9b680739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 -- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68103030 msgr2=0x7f9b68073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68103030 0x7f9b68073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6d39b700 1 -- 172.21.15.66:0/1473694297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580095f0 con 0x7f9b681014a0 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.512+0000 7f9b6cb9a700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68102690 0x7f9b680739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.513+0000 7f9b6d39b700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b68075330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400e970 tx=0x7f9b6400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.513+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64009790 con 0x7f9b681014a0 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.513+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b6400eec0 con 0x7f9b681014a0 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.513+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b681a15d0 con 0x7f9b681014a0 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.513+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64003c10 con 0x7f9b681014a0 2023-12-11T19:31:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.513+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b681a1b30 con 0x7f9b681014a0 2023-12-11T19:31:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.514+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b68108940 con 0x7f9b681014a0 2023-12-11T19:31:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.516+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9b64003d90 con 0x7f9b681014a0 2023-12-11T19:31:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.518+0000 7f9b5e7fc700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b540a1500 0x7f9b540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.518+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9b64014070 con 0x7f9b681014a0 2023-12-11T19:31:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.518+0000 7f9b6cb9a700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b540a1500 0x7f9b540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:26.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.519+0000 7f9b6cb9a700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b540a1500 0x7f9b540a39c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b580 tx=0x7f9b5801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:26.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.520+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b64096620 con 0x7f9b681014a0 2023-12-11T19:31:26.630 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:31:26 smithi167 podman[139320]: 2023-12-11T19:31:26.630 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:31:26 smithi167 bash[139320]: aa05e653c4a3bfcfc145e40a1ade5e9db2f13afcf78450bed2d2091a535eb984 2023-12-11T19:31:26.630 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:31:26 smithi167 systemd[1]: Started Ceph mds.f for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:26.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.791+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b68068230 con 0x7f9b681014a0 2023-12-11T19:31:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.791+0000 7f9b5e7fc700 1 -- 172.21.15.66:0/1473694297 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7f9b64099c40 con 0x7f9b681014a0 2023-12-11T19:31:26.792 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:26.792 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b540a1500 msgr2=0x7f9b540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b540a1500 0x7f9b540a39c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b580 tx=0x7f9b5801a040 comp rx=0 tx=0).stop 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 msgr2=0x7f9b68075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b68075330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400e970 tx=0x7f9b6400ec80 comp rx=0 tx=0).stop 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 shutdown_connections 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b540a1500 0x7f9b540a39c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b68103030 0x7f9b68073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b681014a0 0x7f9b68075330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 --2- 172.21.15.66:0/1473694297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b68102690 0x7f9b680739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 >> 172.21.15.66:0/1473694297 conn(0x7f9b680fca10 msgr2=0x7f9b68105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 shutdown_connections 2023-12-11T19:31:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:26.794+0000 7f9b6f5ff700 1 -- 172.21.15.66:0/1473694297 wait complete. 2023-12-11T19:31:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 6 2023-12-11T19:31:27.020 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:31:26 smithi167 ceph-mds[139361]: -- 172.21.15.167:0/3669334509 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646b6d2b520 con 0x5646b6d03000 2023-12-11T19:31:27.020 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:31:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-f[139336]: starting mds.f at 2023-12-11T19:31:27.987 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: Deploying daemon mds.c on smithi167 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1473694297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] up:boot 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: fsmap 6 up:standby 2023-12-11T19:31:27.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:27 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: Deploying daemon mds.c on smithi167 2023-12-11T19:31:28.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1473694297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:28.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] up:boot 2023-12-11T19:31:28.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: fsmap 6 up:standby 2023-12-11T19:31:28.064 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:27 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: Deploying daemon mds.c on smithi167 2023-12-11T19:31:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1473694297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:28.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] up:boot 2023-12-11T19:31:28.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: fsmap 6 up:standby 2023-12-11T19:31:28.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:27 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T19:31:28.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:29.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 -- 172.21.15.66:0/1547448074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 msgr2=0x7fbecc1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 --2- 172.21.15.66:0/1547448074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1098e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbec400b0d0 tx=0x7fbec400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 -- 172.21.15.66:0/1547448074 shutdown_connections 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 --2- 172.21.15.66:0/1547448074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1098e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 --2- 172.21.15.66:0/1547448074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 --2- 172.21.15.66:0/1547448074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbecc102690 0x7fbecc102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 -- 172.21.15.66:0/1547448074 >> 172.21.15.66:0/1547448074 conn(0x7fbecc0fca10 msgr2=0x7fbecc0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 -- 172.21.15.66:0/1547448074 shutdown_connections 2023-12-11T19:31:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.037+0000 7fbed1fff700 1 -- 172.21.15.66:0/1547448074 wait complete. 2023-12-11T19:31:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.038+0000 7fbed1fff700 1 Processor -- start 2023-12-11T19:31:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 -- start start 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbecc102690 0x7fbecc199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1a11c0 con 0x7fbecc103030 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1a1300 con 0x7fbecc102690 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbed1fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1a1490 con 0x7fbecc1014a0 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbecb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbecbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbecb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37910/0 (socket says 172.21.15.66:37910) 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbecb7fe700 1 -- 172.21.15.66:0/50772905 learned_addr learned my addr 172.21.15.66:0/50772905 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.039+0000 7fbec3fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbecc102690 0x7fbecc199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecbfff700 1 -- 172.21.15.66:0/50772905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 msgr2=0x7fbecc1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecbfff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecbfff700 1 -- 172.21.15.66:0/50772905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbecc102690 msgr2=0x7fbecc199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecbfff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbecc102690 0x7fbecc199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecbfff700 1 -- 172.21.15.66:0/50772905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4009d00 con 0x7fbecc103030 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecb7fe700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbecbfff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1a0bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbec4007b60 tx=0x7fbec4009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400e040 con 0x7fbecc103030 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbec4003e10 con 0x7fbecc103030 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc1a1710 con 0x7fbecc103030 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec401c8c0 con 0x7fbecc103030 2023-12-11T19:31:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.040+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc1a1b30 con 0x7fbecc103030 2023-12-11T19:31:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.042+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc068230 con 0x7fbecc103030 2023-12-11T19:31:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.044+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fbec4005690 con 0x7fbecc103030 2023-12-11T19:31:29.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.045+0000 7fbec97fa700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbeb80a1480 0x7fbeb80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:29.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.045+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbec40cd7c0 con 0x7fbecc103030 2023-12-11T19:31:29.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.045+0000 7fbecb7fe700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbeb80a1480 0x7fbeb80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.046+0000 7fbecb7fe700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbeb80a1480 0x7fbeb80a3940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0105c0 tx=0x7fbebc00b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:29.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.048+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbec4097340 con 0x7fbecc103030 2023-12-11T19:31:29.271 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:31:28 smithi167 systemd[1]: Starting Ceph mds.c for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.307+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbecc10cdd0 con 0x7fbecc103030 2023-12-11T19:31:29.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.308+0000 7fbec97fa700 1 -- 172.21.15.66:0/50772905 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7fbec409a960 con 0x7fbecc103030 2023-12-11T19:31:29.308 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:29.308 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbeb80a1480 msgr2=0x7fbeb80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbeb80a1480 0x7fbeb80a3940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0105c0 tx=0x7fbebc00b4a0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 msgr2=0x7fbecc1a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1a0bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbec4007b60 tx=0x7fbec4009440 comp rx=0 tx=0).stop 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 shutdown_connections 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbeb80a1480 0x7fbeb80a3940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbecc103030 0x7fbecc1a0bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc1014a0 0x7fbecc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 --2- 172.21.15.66:0/50772905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbecc102690 0x7fbecc199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 >> 172.21.15.66:0/50772905 conn(0x7fbecc0fca10 msgr2=0x7fbecc073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 shutdown_connections 2023-12-11T19:31:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:29.310+0000 7fbed1fff700 1 -- 172.21.15.66:0/50772905 wait complete. 2023-12-11T19:31:29.312 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 7 2023-12-11T19:31:29.737 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:31:29 smithi167 podman[139662]: 2023-12-11T19:31:29.738 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:29 smithi167 ceph-mon[102750]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:29.738 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:29 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/50772905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:30.020 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:31:29 smithi167 ceph-mds[139702]: -- 172.21.15.167:0/434453987 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1f6351520 con 0x55d1f6328800 2023-12-11T19:31:30.021 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:31:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-c[139678]: starting mds.c at 2023-12-11T19:31:30.021 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:31:29 smithi167 bash[139662]: 21f9f29999200703d9e4446331bc5597cf1ee0f1a42b6f56f1542ec9249b8360 2023-12-11T19:31:30.021 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:31:29 smithi167 systemd[1]: Started Ceph mds.c for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:29 smithi066 ceph-mon[97920]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:29 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/50772905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:30.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:29 smithi070 ceph-mon[104044]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:30.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:29 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/50772905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: Deploying daemon mds.h on smithi070 2023-12-11T19:31:31.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.167:6836/2775450491,v1:172.21.15.167:6837/2775450491] up:boot 2023-12-11T19:31:31.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: fsmap 7 up:standby 2023-12-11T19:31:31.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:30 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T19:31:31.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:31.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:31.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:31.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: Deploying daemon mds.h on smithi070 2023-12-11T19:31:31.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.167:6836/2775450491,v1:172.21.15.167:6837/2775450491] up:boot 2023-12-11T19:31:31.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: fsmap 7 up:standby 2023-12-11T19:31:31.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:30 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:31.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: Deploying daemon mds.h on smithi070 2023-12-11T19:31:31.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.167:6836/2775450491,v1:172.21.15.167:6837/2775450491] up:boot 2023-12-11T19:31:31.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: fsmap 7 up:standby 2023-12-11T19:31:31.314 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:30 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T19:31:31.331 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:31.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 -- 172.21.15.66:0/2447228554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe028103030 msgr2=0x7fe0281098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:31.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/2447228554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe028103030 0x7fe0281098e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe020009fc0 tx=0x7fe02000b330 comp rx=0 tx=0).stop 2023-12-11T19:31:31.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 -- 172.21.15.66:0/2447228554 shutdown_connections 2023-12-11T19:31:31.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/2447228554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe028103030 0x7fe0281098e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/2447228554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0281014a0 0x7fe0281018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/2447228554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028102690 0x7fe028102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 -- 172.21.15.66:0/2447228554 >> 172.21.15.66:0/2447228554 conn(0x7fe0280fca10 msgr2=0x7fe0280fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 -- 172.21.15.66:0/2447228554 shutdown_connections 2023-12-11T19:31:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.552+0000 7fe02f5ff700 1 -- 172.21.15.66:0/2447228554 wait complete. 2023-12-11T19:31:31.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.553+0000 7fe02f5ff700 1 Processor -- start 2023-12-11T19:31:31.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 -- start start 2023-12-11T19:31:31.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 0x7fe028199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:31.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe028102690 0x7fe028199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028103030 0x7fe0281a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281a11b0 con 0x7fe0281014a0 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281a12f0 con 0x7fe028103030 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02f5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0281a1480 con 0x7fe028102690 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02d39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 0x7fe028199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02d39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 0x7fe028199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39256/0 (socket says 172.21.15.66:39256) 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02d39b700 1 -- 172.21.15.66:0/3738378223 learned_addr learned my addr 172.21.15.66:0/3738378223 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02db9c700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028103030 0x7fe0281a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.554+0000 7fe02cb9a700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe028102690 0x7fe028199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02db9c700 1 -- 172.21.15.66:0/3738378223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe028102690 msgr2=0x7fe028199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:31.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02db9c700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe028102690 0x7fe028199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02db9c700 1 -- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 msgr2=0x7fe028199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02db9c700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 0x7fe028199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02db9c700 1 -- 172.21.15.66:0/3738378223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180095f0 con 0x7fe028103030 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02d39b700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 0x7fe028199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02db9c700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028103030 0x7fe0281a0be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b6c0 tx=0x7fe020008c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000e040 con 0x7fe028103030 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020009c40 con 0x7fe028103030 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe02000bc00 con 0x7fe028103030 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020005040 con 0x7fe028103030 2023-12-11T19:31:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.555+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0281a19f0 con 0x7fe028103030 2023-12-11T19:31:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.556+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe028068230 con 0x7fe028103030 2023-12-11T19:31:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.558+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe02001d470 con 0x7fe028103030 2023-12-11T19:31:31.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.560+0000 7fe01e7fc700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe0140a1480 0x7fe0140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:31.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.560+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0200cdc50 con 0x7fe028103030 2023-12-11T19:31:31.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.560+0000 7fe02d39b700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe0140a1480 0x7fe0140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:31.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.560+0000 7fe02d39b700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe0140a1480 0x7fe0140a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe024005950 tx=0x7fe024009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.564+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe020097740 con 0x7fe028103030 2023-12-11T19:31:31.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.820+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe02810cbb0 con 0x7fe028103030 2023-12-11T19:31:31.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.821+0000 7fe01e7fc700 1 -- 172.21.15.66:0/3738378223 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7fe02009ad60 con 0x7fe028103030 2023-12-11T19:31:31.821 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:31.822 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-12-11T19:31:31.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe0140a1480 msgr2=0x7fe0140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:31.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe0140a1480 0x7fe0140a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe024005950 tx=0x7fe024009260 comp rx=0 tx=0).stop 2023-12-11T19:31:31.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028103030 msgr2=0x7fe0281a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:31.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028103030 0x7fe0281a0be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b6c0 tx=0x7fe020008c80 comp rx=0 tx=0).stop 2023-12-11T19:31:31.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 shutdown_connections 2023-12-11T19:31:31.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe0140a1480 0x7fe0140a3940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe028103030 0x7fe0281a0be0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0281014a0 0x7fe028199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 --2- 172.21.15.66:0/3738378223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe028102690 0x7fe028199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 >> 172.21.15.66:0/3738378223 conn(0x7fe0280fca10 msgr2=0x7fe028073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 shutdown_connections 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:31.823+0000 7fe02f5ff700 1 -- 172.21.15.66:0/3738378223 wait complete. 2023-12-11T19:31:31.825 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2023-12-11T19:31:32.164 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:31 smithi070 ceph-mon[104044]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:32.164 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:31 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3738378223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:31 smithi167 ceph-mon[102750]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:32.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:31 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3738378223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:32.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:31 smithi066 ceph-mon[97920]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:32.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:31 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3738378223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:32.610 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:31:32 smithi070 systemd[1]: Starting Ceph mds.h for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:32.938 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:31:32 smithi070 podman[139826]: 2023-12-11T19:31:32.938 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:31:32 smithi070 bash[139826]: 2f1c6a06191febb4b2fe69c9230d28391d1c820859fc785524ef2170bd9d547c 2023-12-11T19:31:32.938 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:31:32 smithi070 ceph-mds[139866]: -- 172.21.15.70:0/3496701566 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594c680d520 con 0x5594c67e4c00 2023-12-11T19:31:32.938 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:31:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-h[139842]: starting mds.h at 2023-12-11T19:31:33.360 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:31:32 smithi070 systemd[1]: Started Ceph mds.h for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:33.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 -- 172.21.15.66:0/1000420945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 msgr2=0x7f88c4102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 --2- 172.21.15.66:0/1000420945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4102af0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f88b40099c0 tx=0x7f88b4009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 -- 172.21.15.66:0/1000420945 shutdown_connections 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 --2- 172.21.15.66:0/1000420945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c4103030 0x7f88c41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 --2- 172.21.15.66:0/1000420945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c41014a0 0x7f88c41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 --2- 172.21.15.66:0/1000420945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4102af0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 -- 172.21.15.66:0/1000420945 >> 172.21.15.66:0/1000420945 conn(0x7f88c40fca10 msgr2=0x7f88c40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 -- 172.21.15.66:0/1000420945 shutdown_connections 2023-12-11T19:31:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.063+0000 7f88cbfff700 1 -- 172.21.15.66:0/1000420945 wait complete. 2023-12-11T19:31:34.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.064+0000 7f88cbfff700 1 Processor -- start 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.064+0000 7f88cbfff700 1 -- start start 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.064+0000 7f88cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c41014a0 0x7f88c4199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.064+0000 7f88cbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88cbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c4103030 0x7f88c41a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88cbfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c41a1030 con 0x7f88c4102690 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88cbfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c41a1170 con 0x7f88c41014a0 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88cbfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c41a1300 con 0x7f88c4103030 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60570/0 (socket says 172.21.15.66:60570) 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 -- 172.21.15.66:0/2074842810 learned_addr learned my addr 172.21.15.66:0/2074842810 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c9d9b700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c41014a0 0x7f88c4199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 -- 172.21.15.66:0/2074842810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c4103030 msgr2=0x7f88c41a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c4103030 0x7f88c41a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 -- 172.21.15.66:0/2074842810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c41014a0 msgr2=0x7f88c4199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c41014a0 0x7f88c4199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 -- 172.21.15.66:0/2074842810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b40095f0 con 0x7f88c4102690 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c9d9b700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c41014a0 0x7f88c4199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.065+0000 7f88c959a700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4199a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005910 tx=0x7f88b4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.066+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b401d070 con 0x7f88c4102690 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.066+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88b400bd30 con 0x7f88c4102690 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.066+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c41a1580 con 0x7f88c4102690 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.066+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4022dc0 con 0x7f88c4102690 2023-12-11T19:31:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.066+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c41a19a0 con 0x7f88c4102690 2023-12-11T19:31:34.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.067+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c40682a0 con 0x7f88c4102690 2023-12-11T19:31:34.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.069+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f88b4004010 con 0x7f88c4102690 2023-12-11T19:31:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.070+0000 7f88baffd700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88b00a1480 0x7f88b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.070+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88b40cda40 con 0x7f88c4102690 2023-12-11T19:31:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.071+0000 7f88c9d9b700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88b00a1480 0x7f88b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.071+0000 7f88c9d9b700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88b00a1480 0x7f88b00a3940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f88c0005950 tx=0x7f88c00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:34.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.073+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f88b4097650 con 0x7f88c4102690 2023-12-11T19:31:34.185 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:34.185 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6836/280227891,v1:172.21.15.70:6837/280227891] up:boot 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: fsmap 8 up:standby 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:34.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:33 smithi167 ceph-mon[102750]: Deploying daemon mds.i on smithi167 2023-12-11T19:31:34.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6836/280227891,v1:172.21.15.70:6837/280227891] up:boot 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: fsmap 8 up:standby 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:34.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:33 smithi066 ceph-mon[97920]: Deploying daemon mds.i on smithi167 2023-12-11T19:31:34.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.336+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88c40682a0 con 0x7f88c4102690 2023-12-11T19:31:34.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.337+0000 7f88baffd700 1 -- 172.21.15.66:0/2074842810 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7f88c40682a0 con 0x7f88c4102690 2023-12-11T19:31:34.338 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:34.338 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88b00a1480 msgr2=0x7f88b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88b00a1480 0x7f88b00a3940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f88c0005950 tx=0x7f88c00058e0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 msgr2=0x7f88c4199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4199a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005910 tx=0x7f88b4005b00 comp rx=0 tx=0).stop 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 shutdown_connections 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88b00a1480 0x7f88b00a3940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88c41014a0 0x7f88c4199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88c4103030 0x7f88c41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 --2- 172.21.15.66:0/2074842810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c4102690 0x7f88c4199a40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:34.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 >> 172.21.15.66:0/2074842810 conn(0x7f88c40fca10 msgr2=0x7f88c4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:34.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 shutdown_connections 2023-12-11T19:31:34.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:34.340+0000 7f88cbfff700 1 -- 172.21.15.66:0/2074842810 wait complete. 2023-12-11T19:31:34.342 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6836/280227891,v1:172.21.15.70:6837/280227891] up:boot 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: fsmap 8 up:standby 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:34.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:34.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:33 smithi070 ceph-mon[104044]: Deploying daemon mds.i on smithi167 2023-12-11T19:31:35.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:34 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2074842810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:35.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:34 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2074842810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:35.352 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:31:35 smithi167 systemd[1]: Starting Ceph mds.i for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:35.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:34 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2074842810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:36.126 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:35 smithi167 ceph-mon[102750]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:36.126 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:31:35 smithi167 podman[140001]: 2023-12-11T19:31:36.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:35 smithi066 ceph-mon[97920]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:36.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:36.362 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:35 smithi070 ceph-mon[104044]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:36.520 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:31:36 smithi167 bash[140001]: 6f60c3f01b7a77c05df50f700778ebd7d5458e9008a73eaf96749f6e00197ab8 2023-12-11T19:31:36.521 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:31:36 smithi167 ceph-mds[140041]: -- 172.21.15.167:0/2973792587 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d9a02d520 con 0x560d9a005000 2023-12-11T19:31:36.521 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:31:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-i[140017]: starting mds.i at 2023-12-11T19:31:36.521 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:31:36 smithi167 systemd[1]: Started Ceph mds.i for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 -- 172.21.15.66:0/398786754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7300ffcf0 msgr2=0x7fa730102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 --2- 172.21.15.66:0/398786754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7300ffcf0 0x7fa730102120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0098f0 tx=0x7fa72c009c00 comp rx=0 tx=0).stop 2023-12-11T19:31:36.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 -- 172.21.15.66:0/398786754 shutdown_connections 2023-12-11T19:31:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 --2- 172.21.15.66:0/398786754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7300ffcf0 0x7fa730102120 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 --2- 172.21.15.66:0/398786754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 --2- 172.21.15.66:0/398786754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa730102660 0x7fa730104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 -- 172.21.15.66:0/398786754 >> 172.21.15.66:0/398786754 conn(0x7fa7300fb8f0 msgr2=0x7fa7300fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.579+0000 7fa7381ff700 1 -- 172.21.15.66:0/398786754 shutdown_connections 2023-12-11T19:31:36.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.580+0000 7fa7381ff700 1 -- 172.21.15.66:0/398786754 wait complete. 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.580+0000 7fa7381ff700 1 Processor -- start 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 -- start start 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7300ffcf0 0x7fa73019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa730102660 0x7fa73019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301a54a0 con 0x7fa730105030 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301a55e0 con 0x7fa730102660 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa7381ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301a5770 con 0x7fa7300ffcf0 2023-12-11T19:31:36.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa73679c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa73679c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60580/0 (socket says 172.21.15.66:60580) 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa73679c700 1 -- 172.21.15.66:0/2549553232 learned_addr learned my addr 172.21.15.66:0/2549553232 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.581+0000 7fa73579a700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa730102660 0x7fa73019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa73679c700 1 -- 172.21.15.66:0/2549553232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7300ffcf0 msgr2=0x7fa73019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa73679c700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7300ffcf0 0x7fa73019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa73679c700 1 -- 172.21.15.66:0/2549553232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa730102660 msgr2=0x7fa73019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa73679c700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa730102660 0x7fa73019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa73679c700 1 -- 172.21.15.66:0/2549553232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c009520 con 0x7fa730105030 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa73679c700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301a4ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c110 tx=0x7fa72800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728017070 con 0x7fa730105030 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa72800e050 con 0x7fa730105030 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728007880 con 0x7fa730105030 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7301a59f0 con 0x7fa730105030 2023-12-11T19:31:36.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.582+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7301a5eb0 con 0x7fa730105030 2023-12-11T19:31:36.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.584+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7300fd470 con 0x7fa730105030 2023-12-11T19:31:36.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.585+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa72800cad0 con 0x7fa730105030 2023-12-11T19:31:36.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.587+0000 7fa726ffd700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa71c0a1500 0x7fa71c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:36.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.587+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7280cc380 con 0x7fa730105030 2023-12-11T19:31:36.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.587+0000 7fa735f9b700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa71c0a1500 0x7fa71c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:36.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.588+0000 7fa735f9b700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa71c0a1500 0x7fa71c0a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009230 tx=0x7fa72c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:36.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.591+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa728095de0 con 0x7fa730105030 2023-12-11T19:31:36.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.876+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7301133a0 con 0x7fa730105030 2023-12-11T19:31:36.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.876+0000 7fa726ffd700 1 -- 172.21.15.66:0/2549553232 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7fa728099400 con 0x7fa730105030 2023-12-11T19:31:36.877 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:36.877 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa71c0a1500 msgr2=0x7fa71c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa71c0a1500 0x7fa71c0a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009230 tx=0x7fa72c0058e0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 msgr2=0x7fa7301a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301a4ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c110 tx=0x7fa72800c420 comp rx=0 tx=0).stop 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 shutdown_connections 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa71c0a1500 0x7fa71c0a39c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa730105030 0x7fa7301a4ed0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7300ffcf0 0x7fa73019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 --2- 172.21.15.66:0/2549553232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa730102660 0x7fa73019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 >> 172.21.15.66:0/2549553232 conn(0x7fa7300fb8f0 msgr2=0x7fa730104580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 shutdown_connections 2023-12-11T19:31:36.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:36.879+0000 7fa7381ff700 1 -- 172.21.15.66:0/2549553232 wait complete. 2023-12-11T19:31:36.881 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2023-12-11T19:31:37.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: Deploying daemon mds.g on smithi066 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2549553232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] up:boot 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: fsmap 9 up:standby 2023-12-11T19:31:37.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:37 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T19:31:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:37.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:37.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:37.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: Deploying daemon mds.g on smithi066 2023-12-11T19:31:37.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2549553232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:37.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] up:boot 2023-12-11T19:31:37.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: fsmap 9 up:standby 2023-12-11T19:31:37.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:37 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T19:31:37.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:37.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: Deploying daemon mds.g on smithi066 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2549553232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] up:boot 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: fsmap 9 up:standby 2023-12-11T19:31:37.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:37 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T19:31:38.901 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:38.973 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:31:38 smithi066 systemd[1]: Starting Ceph mds.g for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:39.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 -- 172.21.15.66:0/2355611826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb41014b0 msgr2=0x7f9eb41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:39.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/2355611826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb41014b0 0x7f9eb41018d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40099c0 tx=0x7f9ea4009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 -- 172.21.15.66:0/2355611826 shutdown_connections 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/2355611826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb4103040 0x7f9eb41098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/2355611826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb41014b0 0x7f9eb41018d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/2355611826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 0x7f9eb4102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 -- 172.21.15.66:0/2355611826 >> 172.21.15.66:0/2355611826 conn(0x7f9eb40fca20 msgr2=0x7f9eb40fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 -- 172.21.15.66:0/2355611826 shutdown_connections 2023-12-11T19:31:39.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.129+0000 7f9eba5ff700 1 -- 172.21.15.66:0/2355611826 wait complete. 2023-12-11T19:31:39.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.130+0000 7f9eba5ff700 1 Processor -- start 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.130+0000 7f9eba5ff700 1 -- start start 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eba5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb41014b0 0x7f9eb4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eba5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 0x7f9eb4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eba5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 0x7f9eb419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eba5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419ee50 con 0x7f9eb4103040 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eba5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419ef90 con 0x7f9eb41026a0 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eba5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb419f120 con 0x7f9eb41014b0 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 0x7f9eb419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 0x7f9eb419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60602/0 (socket says 172.21.15.66:60602) 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 -- 172.21.15.66:0/4054279441 learned_addr learned my addr 172.21.15.66:0/4054279441 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb3fff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb41014b0 0x7f9eb4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb37fe700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 0x7f9eb4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:39.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 -- 172.21.15.66:0/4054279441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb41014b0 msgr2=0x7f9eb4197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb41014b0 0x7f9eb4197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 -- 172.21.15.66:0/4054279441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 msgr2=0x7f9eb4197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 0x7f9eb4197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb8b9c700 1 -- 172.21.15.66:0/4054279441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40095f0 con 0x7f9eb4103040 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb37fe700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 0x7f9eb4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.131+0000 7f9eb3fff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb41014b0 0x7f9eb4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.132+0000 7f9eb8b9c700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 0x7f9eb419e880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00ef30 tx=0x7f9eac00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.132+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac018070 con 0x7f9eb4103040 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.132+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9eac00f040 con 0x7f9eb4103040 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.132+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac003d00 con 0x7f9eb4103040 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.132+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb419f3a0 con 0x7f9eb4103040 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.132+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb419f860 con 0x7f9eb4103040 2023-12-11T19:31:39.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.133+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb4108980 con 0x7f9eb4103040 2023-12-11T19:31:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.135+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9eac00caa0 con 0x7f9eb4103040 2023-12-11T19:31:39.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.138+0000 7f9eb17fa700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9e9c0a1500 0x7f9e9c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:39.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.138+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9eac0ccf60 con 0x7f9eb4103040 2023-12-11T19:31:39.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.138+0000 7f9eb3fff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9e9c0a1500 0x7f9e9c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:39.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.139+0000 7f9eb3fff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9e9c0a1500 0x7f9e9c0a39c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006010 tx=0x7f9ea4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.140+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9eac096a50 con 0x7f9eb4103040 2023-12-11T19:31:39.307 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:31:39 smithi066 podman[144834]: 2023-12-11T19:31:39.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.415+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb4068230 con 0x7f9eb4103040 2023-12-11T19:31:39.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.416+0000 7f9eb17fa700 1 -- 172.21.15.66:0/4054279441 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7f9eac09a070 con 0x7f9eb4103040 2023-12-11T19:31:39.416 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:39.417 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-11T19:31:39.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9e9c0a1500 msgr2=0x7f9e9c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9e9c0a1500 0x7f9e9c0a39c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006010 tx=0x7f9ea4000bc0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 msgr2=0x7f9eb419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 0x7f9eb419e880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00ef30 tx=0x7f9eac00c450 comp rx=0 tx=0).stop 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 shutdown_connections 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9e9c0a1500 0x7f9e9c0a39c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9eb4103040 0x7f9eb419e880 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eb41014b0 0x7f9eb4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 --2- 172.21.15.66:0/4054279441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb41026a0 0x7f9eb4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 >> 172.21.15.66:0/4054279441 conn(0x7f9eb40fca20 msgr2=0x7f9eb4105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:39.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 shutdown_connections 2023-12-11T19:31:39.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:39.420+0000 7f9eba5ff700 1 -- 172.21.15.66:0/4054279441 wait complete. 2023-12-11T19:31:39.422 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 10 2023-12-11T19:31:39.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:39 smithi066 ceph-mon[97920]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:39.563 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:31:39 smithi066 bash[144834]: 62b8fae8fd6b881fa9b24c541953e56306cc2ff803ec857a3b9239e14be0cd74 2023-12-11T19:31:39.563 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:31:39 smithi066 ceph-mds[144896]: -- 172.21.15.66:0/3748362411 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f26f15d520 con 0x55f26f134c00 2023-12-11T19:31:39.563 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:31:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-g[144870]: starting mds.g at 2023-12-11T19:31:39.563 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:31:39 smithi066 systemd[1]: Started Ceph mds.g for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:39.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:39 smithi070 ceph-mon[104044]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:39.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:39 smithi167 ceph-mon[102750]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4054279441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:40.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:40 smithi070 ceph-mon[104044]: Deploying daemon mds.a on smithi066 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4054279441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:40.625 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:40 smithi066 ceph-mon[97920]: Deploying daemon mds.a on smithi066 2023-12-11T19:31:40.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4054279441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:40.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:40.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:40 smithi167 ceph-mon[102750]: Deploying daemon mds.a on smithi066 2023-12-11T19:31:41.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:41.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:41 smithi070 ceph-mon[104044]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:41.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:41 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.66:6838/2148751938,v1:172.21.15.66:6839/2148751938] up:boot 2023-12-11T19:31:41.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:41 smithi070 ceph-mon[104044]: fsmap 10 up:standby 2023-12-11T19:31:41.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:41 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T19:31:41.669 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:41 smithi066 ceph-mon[97920]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:41.669 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:41 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.66:6838/2148751938,v1:172.21.15.66:6839/2148751938] up:boot 2023-12-11T19:31:41.669 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:41 smithi066 ceph-mon[97920]: fsmap 10 up:standby 2023-12-11T19:31:41.669 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:41 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T19:31:41.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:41 smithi167 ceph-mon[102750]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:41.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:41 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.66:6838/2148751938,v1:172.21.15.66:6839/2148751938] up:boot 2023-12-11T19:31:41.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:41 smithi167 ceph-mon[102750]: fsmap 10 up:standby 2023-12-11T19:31:41.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:41 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T19:31:41.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/3312907477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8105030 msgr2=0x7fe2e81054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/3312907477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8105030 0x7fe2e81054b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000b0a0 tx=0x7fe2e000b3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/3312907477 shutdown_connections 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/3312907477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8105030 0x7fe2e81054b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/3312907477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 0x7fe2e8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/3312907477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2e8102660 0x7fe2e8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/3312907477 >> 172.21.15.66:0/3312907477 conn(0x7fe2e80fb8f0 msgr2=0x7fe2e80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:41.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/3312907477 shutdown_connections 2023-12-11T19:31:41.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.833+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/3312907477 wait complete. 2023-12-11T19:31:41.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 Processor -- start 2023-12-11T19:31:41.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 -- start start 2023-12-11T19:31:41.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 0x7fe2e819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:41.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8102660 0x7fe2e819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:41.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2e8105030 0x7fe2e81a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e81a5580 con 0x7fe2e8102660 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e81a56c0 con 0x7fe2e8105030 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.835+0000 7fe2ed3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e81a5850 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e67fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8102660 0x7fe2e819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e67fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8102660 0x7fe2e819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60636/0 (socket says 172.21.15.66:60636) 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e67fc700 1 -- 172.21.15.66:0/2109784843 learned_addr learned my addr 172.21.15.66:0/2109784843 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e6ffd700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 0x7fe2e819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:41.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e77fe700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2e8105030 0x7fe2e81a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e6ffd700 1 -- 172.21.15.66:0/2109784843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2e8105030 msgr2=0x7fe2e81a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e6ffd700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2e8105030 0x7fe2e81a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e6ffd700 1 -- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8102660 msgr2=0x7fe2e819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e6ffd700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8102660 0x7fe2e819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.836+0000 7fe2e6ffd700 1 -- 172.21.15.66:0/2109784843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0009d00 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.837+0000 7fe2e6ffd700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 0x7fe2e819daa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000d950 tx=0x7fe2d000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.837+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000f890 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.837+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2d000fe80 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.837+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e81a5b30 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.837+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000e3b0 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.837+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e81a6040 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.838+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e80fd470 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.840+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe2d000f9f0 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.841+0000 7fe2dffff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2d40a1480 0x7fe2d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:41.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.842+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2d00cc700 con 0x7fe2e80ffcf0 2023-12-11T19:31:41.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.842+0000 7fe2e67fc700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2d40a1480 0x7fe2d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:41.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.842+0000 7fe2e67fc700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2d40a1480 0x7fe2d40a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8007c90 tx=0x7fe2d8007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:41.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:41.844+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe2d0096160 con 0x7fe2e80ffcf0 2023-12-11T19:31:42.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.109+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2e8068230 con 0x7fe2e80ffcf0 2023-12-11T19:31:42.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.110+0000 7fe2dffff700 1 -- 172.21.15.66:0/2109784843 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8456 (secure 0 0 0) 0x7fe2d0099780 con 0x7fe2e80ffcf0 2023-12-11T19:31:42.114 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:42.115 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14673,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2d40a1480 msgr2=0x7fe2d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2d40a1480 0x7fe2d40a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8007c90 tx=0x7fe2d8007480 comp rx=0 tx=0).stop 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 msgr2=0x7fe2e819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 0x7fe2e819daa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000d950 tx=0x7fe2d000dc60 comp rx=0 tx=0).stop 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 shutdown_connections 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe2d40a1480 0x7fe2d40a3940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2e8105030 0x7fe2e81a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e80ffcf0 0x7fe2e819daa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 --2- 172.21.15.66:0/2109784843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2e8102660 0x7fe2e819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 >> 172.21.15.66:0/2109784843 conn(0x7fe2e80fb8f0 msgr2=0x7fe2e8100670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 shutdown_connections 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:42.114+0000 7fe2ed3ff700 1 -- 172.21.15.66:0/2109784843 wait complete. 2023-12-11T19:31:42.116 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 11 2023-12-11T19:31:42.312 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:31:42 smithi066 systemd[1]: Starting Ceph mds.a for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:42.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:42 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2109784843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:42.808 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:31:42 smithi066 podman[145247]: 2023-12-11T19:31:42.808 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:42 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2109784843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:42.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:42 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2109784843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:43.062 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:31:42 smithi066 bash[145247]: c8a30422417badce70ebb25fbcaca472c2425ab6483bf1561dbc797c3cd40579 2023-12-11T19:31:43.062 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:31:42 smithi066 ceph-mds[145287]: -- 172.21.15.66:0/3178955646 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6f0ef7520 con 0x55b6f0ecf000 2023-12-11T19:31:43.062 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:31:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-a[145263]: starting mds.a at 2023-12-11T19:31:43.063 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:31:42 smithi066 systemd[1]: Started Ceph mds.a for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:44.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:44.171 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: Deploying daemon mds.b on smithi070 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.66:6840/1874182059,v1:172.21.15.66:6841/1874182059] up:boot 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: fsmap 11 up:standby 2023-12-11T19:31:44.172 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:43 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T19:31:44.215 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: Deploying daemon mds.b on smithi070 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.66:6840/1874182059,v1:172.21.15.66:6841/1874182059] up:boot 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: fsmap 11 up:standby 2023-12-11T19:31:44.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:43 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T19:31:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: Deploying daemon mds.b on smithi070 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.66:6840/1874182059,v1:172.21.15.66:6841/1874182059] up:boot 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: fsmap 11 up:standby 2023-12-11T19:31:44.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:43 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T19:31:44.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 -- 172.21.15.66:0/480152207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4103030 msgr2=0x7f9ff41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:44.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/480152207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4103030 0x7f9ff41098e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b0d0 tx=0x7f9fec00b3e0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 -- 172.21.15.66:0/480152207 shutdown_connections 2023-12-11T19:31:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/480152207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 0x7f9ff41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/480152207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4103030 0x7f9ff41098e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/480152207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff4102690 0x7f9ff4102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.365+0000 7f9ff95ff700 1 -- 172.21.15.66:0/480152207 >> 172.21.15.66:0/480152207 conn(0x7f9ff40fca10 msgr2=0x7f9ff40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.366+0000 7f9ff95ff700 1 -- 172.21.15.66:0/480152207 shutdown_connections 2023-12-11T19:31:44.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.366+0000 7f9ff95ff700 1 -- 172.21.15.66:0/480152207 wait complete. 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 Processor -- start 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 -- start start 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 0x7f9ff4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 0x7f9ff4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff4103030 0x7f9ff419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419ef60 con 0x7f9ff4102690 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419f0a0 con 0x7f9ff41014a0 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff95ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419f230 con 0x7f9ff4103030 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 0x7f9ff4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:44.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 0x7f9ff4197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40570/0 (socket says 172.21.15.66:40570) 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff27fc700 1 -- 172.21.15.66:0/1158050650 learned_addr learned my addr 172.21.15.66:0/1158050650 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff2ffd700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 0x7f9ff4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.367+0000 7f9ff37fe700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff4103030 0x7f9ff419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff2ffd700 1 -- 172.21.15.66:0/1158050650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff4103030 msgr2=0x7f9ff419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff2ffd700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff4103030 0x7f9ff419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff2ffd700 1 -- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 msgr2=0x7f9ff4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff2ffd700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 0x7f9ff4197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff2ffd700 1 -- 172.21.15.66:0/1158050650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009d00 con 0x7f9ff41014a0 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff27fc700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 0x7f9ff4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff2ffd700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 0x7f9ff4197430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400e9c0 tx=0x7f9fe400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004220 con 0x7f9ff41014a0 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff419f510 con 0x7f9ff41014a0 2023-12-11T19:31:44.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fe4004380 con 0x7f9ff41014a0 2023-12-11T19:31:44.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.368+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003b40 con 0x7f9ff41014a0 2023-12-11T19:31:44.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.369+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff419fa70 con 0x7f9ff41014a0 2023-12-11T19:31:44.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.369+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff4108970 con 0x7f9ff41014a0 2023-12-11T19:31:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.372+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9fe4003ca0 con 0x7f9ff41014a0 2023-12-11T19:31:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.373+0000 7f9fdbfff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fdc0a1480 0x7f9fdc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:44.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.373+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9fe4014070 con 0x7f9ff41014a0 2023-12-11T19:31:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.373+0000 7f9ff27fc700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fdc0a1480 0x7f9fdc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:44.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.374+0000 7f9ff27fc700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fdc0a1480 0x7f9fdc0a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005950 tx=0x7f9fe800b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:44.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.376+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9fe40966e0 con 0x7f9ff41014a0 2023-12-11T19:31:44.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.636+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ff4068230 con 0x7f9ff41014a0 2023-12-11T19:31:44.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.637+0000 7f9fdbfff700 1 -- 172.21.15.66:0/1158050650 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9248 (secure 0 0 0) 0x7f9fe4099d00 con 0x7f9ff41014a0 2023-12-11T19:31:44.637 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:44.638 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14673,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14685,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-11T19:31:44.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.639+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fdc0a1480 msgr2=0x7f9fdc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.639+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fdc0a1480 0x7f9fdc0a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005950 tx=0x7f9fe800b500 comp rx=0 tx=0).stop 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.639+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 msgr2=0x7f9ff4197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.639+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 0x7f9ff4197430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400e9c0 tx=0x7f9fe400ecd0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 shutdown_connections 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fdc0a1480 0x7f9fdc0a3940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ff41014a0 0x7f9ff4197430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff4103030 0x7f9ff419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 --2- 172.21.15.66:0/1158050650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ff4102690 0x7f9ff4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 >> 172.21.15.66:0/1158050650 conn(0x7f9ff40fca10 msgr2=0x7f9ff4105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 shutdown_connections 2023-12-11T19:31:44.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:44.640+0000 7f9ff95ff700 1 -- 172.21.15.66:0/1158050650 wait complete. 2023-12-11T19:31:44.642 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 12 2023-12-11T19:31:45.121 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:44 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1158050650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:45.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:44 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1158050650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:45.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:44 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1158050650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:45.610 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:45 smithi070 systemd[1]: Starting Ceph mds.b for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:31:45.860 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:45 smithi070 podman[140178]: 2023-12-11T19:31:45.860 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:45 smithi070 bash[140178]: e0de8f6167d307e457eb1952e2809f9a8f245c6f1b8b9b089c4f3a9474819f9f 2023-12-11T19:31:46.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:45 smithi167 ceph-mon[102750]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:46.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:45 smithi066 ceph-mon[97920]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:46.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:45 smithi070 ceph-mon[104044]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:46.360 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:45 smithi070 ceph-mds[140218]: -- 172.21.15.70:0/69512970 <== mon.2 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f660b3520 con 0x559f6608ac00 2023-12-11T19:31:46.360 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b[140194]: starting mds.b at 2023-12-11T19:31:46.360 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:45 smithi070 systemd[1]: Started Ceph mds.b for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:31:46.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 -- 172.21.15.66:0/630332105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 msgr2=0x7f4900102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 --2- 172.21.15.66:0/630332105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 0x7f4900102af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009920 tx=0x7f48f0009c30 comp rx=0 tx=0).stop 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 -- 172.21.15.66:0/630332105 shutdown_connections 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 --2- 172.21.15.66:0/630332105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4900103030 0x7f49001098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 --2- 172.21.15.66:0/630332105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49001014a0 0x7f49001018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 --2- 172.21.15.66:0/630332105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 0x7f4900102af0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 -- 172.21.15.66:0/630332105 >> 172.21.15.66:0/630332105 conn(0x7f49000fca10 msgr2=0x7f49000fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 -- 172.21.15.66:0/630332105 shutdown_connections 2023-12-11T19:31:46.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.883+0000 7f49087ff700 1 -- 172.21.15.66:0/630332105 wait complete. 2023-12-11T19:31:46.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.884+0000 7f49087ff700 1 Processor -- start 2023-12-11T19:31:46.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 -- start start 2023-12-11T19:31:46.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 0x7f4900075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 0x7f49000739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4900103030 0x7f4900073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000746a0 con 0x7f49001014a0 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4900074810 con 0x7f4900102690 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f49087ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a1440 con 0x7f4900103030 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f490659b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 0x7f4900075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f490659b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 0x7f4900075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40578/0 (socket says 172.21.15.66:40578) 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f490659b700 1 -- 172.21.15.66:0/1047865976 learned_addr learned my addr 172.21.15.66:0/1047865976 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.885+0000 7f4905d9a700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 0x7f49000739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f4906d9c700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4900103030 0x7f4900073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f490659b700 1 -- 172.21.15.66:0/1047865976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4900103030 msgr2=0x7f4900073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f490659b700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4900103030 0x7f4900073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f490659b700 1 -- 172.21.15.66:0/1047865976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 msgr2=0x7f49000739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:46.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f490659b700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 0x7f49000739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f490659b700 1 -- 172.21.15.66:0/1047865976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0009520 con 0x7f49001014a0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f490659b700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 0x7f4900075330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00d980 tx=0x7f48fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc004020 con 0x7f49001014a0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48fc0096d0 con 0x7f49001014a0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49001a16d0 con 0x7f49001014a0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003a00 con 0x7f49001014a0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.886+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49001a1c30 con 0x7f49001014a0 2023-12-11T19:31:46.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.887+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900108960 con 0x7f49001014a0 2023-12-11T19:31:46.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.889+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f48fc009840 con 0x7f49001014a0 2023-12-11T19:31:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.891+0000 7f48f77fe700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48ec0a1500 0x7f48ec0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.891+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48fc0cc4d0 con 0x7f49001014a0 2023-12-11T19:31:46.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.891+0000 7f4905d9a700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48ec0a1500 0x7f48ec0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:46.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.892+0000 7f4905d9a700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48ec0a1500 0x7f48ec0a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48f000f9d0 tx=0x7f48f0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:46.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:46.895+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f48fc095f30 con 0x7f49001014a0 2023-12-11T19:31:47.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] up:boot 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: fsmap 12 up:standby 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:46 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:47.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.160+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4900068230 con 0x7f49001014a0 2023-12-11T19:31:47.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.161+0000 7f48f77fe700 1 -- 172.21.15.66:0/1047865976 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10037 (secure 0 0 0) 0x7f48fc099550 con 0x7f49001014a0 2023-12-11T19:31:47.162 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:47.162 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14673,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14685,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24533,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-12-11T19:31:47.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48ec0a1500 msgr2=0x7f48ec0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48ec0a1500 0x7f48ec0a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48f000f9d0 tx=0x7f48f0019040 comp rx=0 tx=0).stop 2023-12-11T19:31:47.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 msgr2=0x7f4900075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 0x7f4900075330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00d980 tx=0x7f48fc00dc90 comp rx=0 tx=0).stop 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 shutdown_connections 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48ec0a1500 0x7f48ec0a39c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4900103030 0x7f4900073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49001014a0 0x7f4900075330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 --2- 172.21.15.66:0/1047865976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4900102690 0x7f49000739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.165+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 >> 172.21.15.66:0/1047865976 conn(0x7f49000fca10 msgr2=0x7f4900105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.166+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 shutdown_connections 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.166+0000 7f49087ff700 1 -- 172.21.15.66:0/1047865976 wait complete. 2023-12-11T19:31:47.167 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 13 2023-12-11T19:31:47.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:31:47.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] up:boot 2023-12-11T19:31:47.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: fsmap 12 up:standby 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:46 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] up:boot 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: fsmap 12 up:standby 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:47.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:46 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 -- 172.21.15.66:0/3444020600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608104830 msgr2=0x7f3608106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 --2- 172.21.15.66:0/3444020600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608104830 0x7f3608106c20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009990 tx=0x7f35f8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 -- 172.21.15.66:0/3444020600 shutdown_connections 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 --2- 172.21.15.66:0/3444020600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608101f00 0x7f36081042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 --2- 172.21.15.66:0/3444020600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3608107160 0x7f36081095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 --2- 172.21.15.66:0/3444020600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608104830 0x7f3608106c20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 -- 172.21.15.66:0/3444020600 >> 172.21.15.66:0/3444020600 conn(0x7f36080fb8f0 msgr2=0x7f36080fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:47.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 -- 172.21.15.66:0/3444020600 shutdown_connections 2023-12-11T19:31:47.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.405+0000 7f360f7ff700 1 -- 172.21.15.66:0/3444020600 wait complete. 2023-12-11T19:31:47.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 Processor -- start 2023-12-11T19:31:47.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 -- start start 2023-12-11T19:31:47.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3608101f00 0x7f3608197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608104830 0x7f3608197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 0x7f360819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360819ee40 con 0x7f3608107160 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360819ef80 con 0x7f3608104830 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.407+0000 7f360f7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360819f110 con 0x7f3608101f00 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 0x7f360819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 0x7f360819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40600/0 (socket says 172.21.15.66:40600) 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 -- 172.21.15.66:0/42190541 learned_addr learned my addr 172.21.15.66:0/42190541 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360d59b700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3608101f00 0x7f3608197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360cd9a700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608104830 0x7f3608197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 -- 172.21.15.66:0/42190541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3608101f00 msgr2=0x7f3608197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3608101f00 0x7f3608197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 -- 172.21.15.66:0/42190541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608104830 msgr2=0x7f3608197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608104830 0x7f3608197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 -- 172.21.15.66:0/42190541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3604009520 con 0x7f3608107160 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360cd9a700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608104830 0x7f3608197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f360dd9c700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 0x7f360819e870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f360000ef30 tx=0x7f360000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.408+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600018070 con 0x7f3608107160 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.409+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f80095f0 con 0x7f3608107160 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.409+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f360000f040 con 0x7f3608107160 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.409+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600003d00 con 0x7f3608107160 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.409+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360819f710 con 0x7f3608107160 2023-12-11T19:31:47.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.410+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36080fd470 con 0x7f3608107160 2023-12-11T19:31:47.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.412+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f360000caa0 con 0x7f3608107160 2023-12-11T19:31:47.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.413+0000 7f35fe7fc700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35f40a1480 0x7f35f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.413+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f36000cc650 con 0x7f3608107160 2023-12-11T19:31:47.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.413+0000 7f360d59b700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35f40a1480 0x7f35f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.414+0000 7f360d59b700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35f40a1480 0x7f35f40a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3604009d80 tx=0x7f3604009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:47.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.417+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3600096140 con 0x7f3608107160 2023-12-11T19:31:47.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.640+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360802d310 con 0x7f3608107160 2023-12-11T19:31:47.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.641+0000 7f35fe7fc700 1 -- 172.21.15.66:0/42190541 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18314 (secure 0 0 0) 0x7f3600099760 con 0x7f3608107160 2023-12-11T19:31:47.642 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:47.642 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":82,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:30:58.331837+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:31:47.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.643+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35f40a1480 msgr2=0x7f35f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.643+0000 7f360f7ff700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35f40a1480 0x7f35f40a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3604009d80 tx=0x7f3604009260 comp rx=0 tx=0).stop 2023-12-11T19:31:47.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.643+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 msgr2=0x7f360819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.643+0000 7f360f7ff700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 0x7f360819e870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f360000ef30 tx=0x7f360000c450 comp rx=0 tx=0).stop 2023-12-11T19:31:47.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 shutdown_connections 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35f40a1480 0x7f35f40a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3608107160 0x7f360819e870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3608101f00 0x7f3608197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 --2- 172.21.15.66:0/42190541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3608104830 0x7f3608197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 >> 172.21.15.66:0/42190541 conn(0x7f36080fb8f0 msgr2=0x7f3608103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 shutdown_connections 2023-12-11T19:31:47.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.644+0000 7f360f7ff700 1 -- 172.21.15.66:0/42190541 wait complete. 2023-12-11T19:31:47.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:31:47.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 -- 172.21.15.66:0/3522769881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b141015a0 msgr2=0x7f2b141019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/3522769881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b141015a0 0x7f2b141019c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b100099c0 tx=0x7f2b10009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 -- 172.21.15.66:0/3522769881 shutdown_connections 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/3522769881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b14103130 0x7f2b141099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/3522769881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b141015a0 0x7f2b141019c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/3522769881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b14102790 0x7f2b14102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 -- 172.21.15.66:0/3522769881 >> 172.21.15.66:0/3522769881 conn(0x7f2b140fcb10 msgr2=0x7f2b140fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:47.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 -- 172.21.15.66:0/3522769881 shutdown_connections 2023-12-11T19:31:47.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.881+0000 7f2b1bdff700 1 -- 172.21.15.66:0/3522769881 wait complete. 2023-12-11T19:31:47.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.882+0000 7f2b1bdff700 1 Processor -- start 2023-12-11T19:31:47.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.882+0000 7f2b1bdff700 1 -- start start 2023-12-11T19:31:47.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1bdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b141015a0 0x7f2b14197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1bdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 0x7f2b14197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1bdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b14103130 0x7f2b1419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1bdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1419f030 con 0x7f2b14102790 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1bdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1419f170 con 0x7f2b141015a0 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1bdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1419f300 con 0x7f2b14103130 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 0x7f2b14197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 0x7f2b14197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40622/0 (socket says 172.21.15.66:40622) 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 -- 172.21.15.66:0/1471343904 learned_addr learned my addr 172.21.15.66:0/1471343904 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b19b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b141015a0 0x7f2b14197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1a39c700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b14103130 0x7f2b1419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 -- 172.21.15.66:0/1471343904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b14103130 msgr2=0x7f2b1419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b14103130 0x7f2b1419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 -- 172.21.15.66:0/1471343904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b141015a0 msgr2=0x7f2b14197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b141015a0 0x7f2b14197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.883+0000 7f2b1939a700 1 -- 172.21.15.66:0/1471343904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c009c40 con 0x7f2b14102790 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.884+0000 7f2b1939a700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 0x7f2b14197a40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400e9a0 tx=0x7f2b0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.884+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04009790 con 0x7f2b14102790 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.884+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b0400ce50 con 0x7f2b14102790 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.884+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b100095f0 con 0x7f2b14102790 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.884+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04018420 con 0x7f2b14102790 2023-12-11T19:31:47.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.884+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1419f9b0 con 0x7f2b14102790 2023-12-11T19:31:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.885+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b14108ac0 con 0x7f2b14102790 2023-12-11T19:31:47.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.887+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2b04010430 con 0x7f2b14102790 2023-12-11T19:31:47.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.888+0000 7f2b0affd700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2b000a1500 0x7f2b000a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:47.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.889+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2b04014070 con 0x7f2b14102790 2023-12-11T19:31:47.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.889+0000 7f2b19b9b700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2b000a1500 0x7f2b000a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:47.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.889+0000 7f2b19b9b700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2b000a1500 0x7f2b000a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009990 tx=0x7f2b10003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:47.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:47.892+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2b040522f0 con 0x7f2b14102790 2023-12-11T19:31:48.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.118+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b141084a0 con 0x7f2b14102790 2023-12-11T19:31:48.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.119+0000 7f2b0affd700 1 -- 172.21.15.66:0/1471343904 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b04096490 con 0x7f2b14102790 2023-12-11T19:31:48.120 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:31:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2b000a1500 msgr2=0x7f2b000a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2b000a1500 0x7f2b000a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009990 tx=0x7f2b10003680 comp rx=0 tx=0).stop 2023-12-11T19:31:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 msgr2=0x7f2b14197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 0x7f2b14197a40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400e9a0 tx=0x7f2b0400ecb0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 shutdown_connections 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2b000a1500 0x7f2b000a39c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b141015a0 0x7f2b14197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b14103130 0x7f2b1419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 --2- 172.21.15.66:0/1471343904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b14102790 0x7f2b14197a40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 >> 172.21.15.66:0/1471343904 conn(0x7f2b140fcb10 msgr2=0x7f2b14105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 shutdown_connections 2023-12-11T19:31:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.122+0000 7f2b1bdff700 1 -- 172.21.15.66:0/1471343904 wait complete. 2023-12-11T19:31:48.147 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-11T19:31:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:48 smithi066 ceph-mon[97920]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:48 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1047865976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:48.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:48 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/42190541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:48.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:48 smithi070 ceph-mon[104044]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:48.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:48 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1047865976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:48.361 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:48 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/42190541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:48.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.361+0000 7f06f87ff700 1 -- 172.21.15.66:0/2942505450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 msgr2=0x7f06f0105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.361+0000 7f06f87ff700 1 --2- 172.21.15.66:0/2942505450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 0x7f06f0105490 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009f90 tx=0x7f06e800b330 comp rx=0 tx=0).stop 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 -- 172.21.15.66:0/2942505450 shutdown_connections 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 --2- 172.21.15.66:0/2942505450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f0102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 --2- 172.21.15.66:0/2942505450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 0x7f06f0105490 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 --2- 172.21.15.66:0/2942505450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 0x7f06f0104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 -- 172.21.15.66:0/2942505450 >> 172.21.15.66:0/2942505450 conn(0x7f06f00fb8f0 msgr2=0x7f06f00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 -- 172.21.15.66:0/2942505450 shutdown_connections 2023-12-11T19:31:48.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.362+0000 7f06f87ff700 1 -- 172.21.15.66:0/2942505450 wait complete. 2023-12-11T19:31:48.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.364+0000 7f06f87ff700 1 Processor -- start 2023-12-11T19:31:48.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.364+0000 7f06f87ff700 1 -- start start 2023-12-11T19:31:48.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.364+0000 7f06f87ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f87ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 0x7f06f019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f87ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 0x7f06f01a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f87ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f01a5530 con 0x7f06f0105010 2023-12-11T19:31:48.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f87ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f01a5670 con 0x7f06f00ffcd0 2023-12-11T19:31:48.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f87ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f01a5800 con 0x7f06f0102640 2023-12-11T19:31:48.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:59618/0 (socket says 172.21.15.66:59618) 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 -- 172.21.15.66:0/588005513 learned_addr learned my addr 172.21.15.66:0/588005513 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f6d9c700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 0x7f06f01a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f5d9a700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 0x7f06f019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 -- 172.21.15.66:0/588005513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 msgr2=0x7f06f019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 0x7f06f019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 -- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 msgr2=0x7f06f01a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 0x7f06f01a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f659b700 1 -- 172.21.15.66:0/588005513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009c40 con 0x7f06f00ffcd0 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.365+0000 7f06f5d9a700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 0x7f06f019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.366+0000 7f06f659b700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f019da50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00e9c0 tx=0x7f06ec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:48.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.366+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00cbd0 con 0x7f06f00ffcd0 2023-12-11T19:31:48.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.366+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06ec00eeb0 con 0x7f06f00ffcd0 2023-12-11T19:31:48.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.366+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f01a5ae0 con 0x7f06f00ffcd0 2023-12-11T19:31:48.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.366+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec005410 con 0x7f06f00ffcd0 2023-12-11T19:31:48.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.366+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f01a5ff0 con 0x7f06f00ffcd0 2023-12-11T19:31:48.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.367+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f00fd470 con 0x7f06f00ffcd0 2023-12-11T19:31:48.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.370+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f06ec0055b0 con 0x7f06f00ffcd0 2023-12-11T19:31:48.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.372+0000 7f06e77fe700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06dc0a1480 0x7f06dc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.372+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06ec014070 con 0x7f06f00ffcd0 2023-12-11T19:31:48.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.372+0000 7f06f5d9a700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06dc0a1480 0x7f06dc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.373+0000 7f06f5d9a700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06dc0a1480 0x7f06dc0a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009cb0 tx=0x7f06e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:48.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.378+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f06ec096630 con 0x7f06f00ffcd0 2023-12-11T19:31:48.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:48 smithi167 ceph-mon[102750]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:48.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:48 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1047865976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:31:48.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:48 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/42190541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:48.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.601+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f06f0068230 con 0x7f06f00ffcd0 2023-12-11T19:31:48.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.602+0000 7f06e77fe700 1 -- 172.21.15.66:0/588005513 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f06ec099c50 con 0x7f06f00ffcd0 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:31:48.603 INFO:teuthology.orchestra.run.smithi066.stdout:] 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06dc0a1480 msgr2=0x7f06dc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06dc0a1480 0x7f06dc0a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009cb0 tx=0x7f06e0009260 comp rx=0 tx=0).stop 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 msgr2=0x7f06f019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f019da50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00e9c0 tx=0x7f06ec00ecd0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 shutdown_connections 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06dc0a1480 0x7f06dc0a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06f00ffcd0 0x7f06f019da50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06f0105010 0x7f06f01a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 --2- 172.21.15.66:0/588005513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06f0102640 0x7f06f019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 >> 172.21.15.66:0/588005513 conn(0x7f06f00fb8f0 msgr2=0x7f06f0100650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:48.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 shutdown_connections 2023-12-11T19:31:48.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.604+0000 7f06f87ff700 1 -- 172.21.15.66:0/588005513 wait complete. 2023-12-11T19:31:48.625 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-11T19:31:48.625 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 -- 172.21.15.66:0/2091433112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8075590 msgr2=0x7f5da80739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 --2- 172.21.15.66:0/2091433112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8075590 0x7f5da80739e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009990 tx=0x7f5d98009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 -- 172.21.15.66:0/2091433112 shutdown_connections 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 --2- 172.21.15.66:0/2091433112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8109490 0x7f5da810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 --2- 172.21.15.66:0/2091433112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8075590 0x7f5da80739e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 --2- 172.21.15.66:0/2091433112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da8073fb0 0x7f5da8074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 -- 172.21.15.66:0/2091433112 >> 172.21.15.66:0/2091433112 conn(0x7f5da80fcb10 msgr2=0x7f5da80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 -- 172.21.15.66:0/2091433112 shutdown_connections 2023-12-11T19:31:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.845+0000 7f5dadbff700 1 -- 172.21.15.66:0/2091433112 wait complete. 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.846+0000 7f5dadbff700 1 Processor -- start 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.846+0000 7f5dadbff700 1 -- start start 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5dadbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8073fb0 0x7f5da8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5dadbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da8075590 0x7f5da8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5dadbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 0x7f5da81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5dadbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da81a11b0 con 0x7f5da8109490 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5dadbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da81a12f0 con 0x7f5da8075590 2023-12-11T19:31:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5dadbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da81a1480 con 0x7f5da8073fb0 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 0x7f5da81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 0x7f5da81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40660/0 (socket says 172.21.15.66:40660) 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 -- 172.21.15.66:0/328315380 learned_addr learned my addr 172.21.15.66:0/328315380 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da6ffd700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da8075590 0x7f5da8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da77fe700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8073fb0 0x7f5da8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 -- 172.21.15.66:0/328315380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8073fb0 msgr2=0x7f5da8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8073fb0 0x7f5da8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 -- 172.21.15.66:0/328315380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da8075590 msgr2=0x7f5da8199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.847+0000 7f5da7fff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da8075590 0x7f5da8199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5da7fff700 1 -- 172.21.15.66:0/328315380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980095f0 con 0x7f5da8109490 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5da77fe700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8073fb0 0x7f5da8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5da7fff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 0x7f5da81a0be0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ef30 tx=0x7f5da000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0018070 con 0x7f5da8109490 2023-12-11T19:31:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5da000f040 con 0x7f5da8109490 2023-12-11T19:31:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da81a1700 con 0x7f5da8109490 2023-12-11T19:31:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da000cc70 con 0x7f5da8109490 2023-12-11T19:31:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.848+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da81a1bc0 con 0x7f5da8109490 2023-12-11T19:31:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.849+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da81082a0 con 0x7f5da8109490 2023-12-11T19:31:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.853+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5da0007500 con 0x7f5da8109490 2023-12-11T19:31:48.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.854+0000 7f5da4ff9700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5d900a1480 0x7f5d900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:48.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.854+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5da00cc470 con 0x7f5da8109490 2023-12-11T19:31:48.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.854+0000 7f5da77fe700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5d900a1480 0x7f5d900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:48.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.855+0000 7f5da77fe700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5d900a1480 0x7f5d900a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d98006010 tx=0x7f5d9800b630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:48.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:48.862+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5da0095f60 con 0x7f5da8109490 2023-12-11T19:31:49.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:49.086+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5da8068310 con 0x7f5da8109490 2023-12-11T19:31:49.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:49 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1471343904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:49.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:49 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/588005513' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-11T19:31:49.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:49 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1471343904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:49.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:49 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/588005513' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-11T19:31:49.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:49 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1471343904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:31:49.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:49 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/588005513' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-11T19:31:50.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.037+0000 7f5da4ff9700 1 -- 172.21.15.66:0/328315380 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5da0099580 con 0x7f5da8109490 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5d900a1480 msgr2=0x7f5d900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5d900a1480 0x7f5d900a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d98006010 tx=0x7f5d9800b630 comp rx=0 tx=0).stop 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 msgr2=0x7f5da81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 0x7f5da81a0be0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5da000ef30 tx=0x7f5da000c3a0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 shutdown_connections 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5d900a1480 0x7f5d900a3940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5da8109490 0x7f5da81a0be0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da8073fb0 0x7f5da8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 --2- 172.21.15.66:0/328315380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5da8075590 0x7f5da8199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 >> 172.21.15.66:0/328315380 conn(0x7f5da80fcb10 msgr2=0x7f5da81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:50.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 shutdown_connections 2023-12-11T19:31:50.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.042+0000 7f5dadbff700 1 -- 172.21.15.66:0/328315380 wait complete. 2023-12-11T19:31:50.045 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2023-12-11T19:31:50.067 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 -- 172.21.15.66:0/1523547599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc074030 msgr2=0x7f75dc0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 --2- 172.21.15.66:0/1523547599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc074030 0x7f75dc0744b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009990 tx=0x7f75d0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 -- 172.21.15.66:0/1523547599 shutdown_connections 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 --2- 172.21.15.66:0/1523547599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc10db70 0x7f75dc10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 --2- 172.21.15.66:0/1523547599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 --2- 172.21.15.66:0/1523547599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc074030 0x7f75dc0744b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 -- 172.21.15.66:0/1523547599 >> 172.21.15.66:0/1523547599 conn(0x7f75dc0fcb10 msgr2=0x7f75dc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:50.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.290+0000 7f75e21ff700 1 -- 172.21.15.66:0/1523547599 shutdown_connections 2023-12-11T19:31:50.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.291+0000 7f75e21ff700 1 -- 172.21.15.66:0/1523547599 wait complete. 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 Processor -- start 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 -- start start 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc074030 0x7f75dc19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc10db70 0x7f75dc1a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a55c0 con 0x7f75dc10db70 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a5700 con 0x7f75dc074030 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75e21ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a5890 con 0x7f75dc075610 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75dbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc10db70 0x7f75dc1a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75daffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.292+0000 7f75daffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:47668/0 (socket says 172.21.15.66:47668) 2023-12-11T19:31:50.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75daffd700 1 -- 172.21.15.66:0/2480447381 learned_addr learned my addr 172.21.15.66:0/2480447381 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:50.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75db7fe700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc074030 0x7f75dc19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75dbfff700 1 -- 172.21.15.66:0/2480447381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 msgr2=0x7f75dc19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75dbfff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75dbfff700 1 -- 172.21.15.66:0/2480447381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc074030 msgr2=0x7f75dc19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75dbfff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc074030 0x7f75dc19da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75dbfff700 1 -- 172.21.15.66:0/2480447381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00095f0 con 0x7f75dc10db70 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75daffd700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75db7fe700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc074030 0x7f75dc19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75dbfff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc10db70 0x7f75dc1a4ff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f75d400efc0 tx=0x7f75d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4018070 con 0x7f75dc10db70 2023-12-11T19:31:50.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc1a5b10 con 0x7f75dc10db70 2023-12-11T19:31:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.293+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d400f040 con 0x7f75dc10db70 2023-12-11T19:31:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.294+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc1a6020 con 0x7f75dc10db70 2023-12-11T19:31:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.294+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400cb80 con 0x7f75dc10db70 2023-12-11T19:31:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.295+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc103dd0 con 0x7f75dc10db70 2023-12-11T19:31:50.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.297+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f75d4003a00 con 0x7f75dc10db70 2023-12-11T19:31:50.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.299+0000 7f75d8ff9700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75c40a1480 0x7f75c40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:50.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.299+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f75d40cce50 con 0x7f75dc10db70 2023-12-11T19:31:50.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.299+0000 7f75db7fe700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75c40a1480 0x7f75c40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:50.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.300+0000 7f75db7fe700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75c40a1480 0x7f75c40a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f75cc010420 tx=0x7f75cc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.302+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f75d40967d0 con 0x7f75dc10db70 2023-12-11T19:31:50.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:50 smithi066 ceph-mon[97920]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:50.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:50 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/328315380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-11T19:31:50.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:50 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:50 smithi070 ceph-mon[104044]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:50 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/328315380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-11T19:31:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:50 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:50 smithi167 ceph-mon[102750]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:50.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:50 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/328315380' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-11T19:31:50.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:50 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:50.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:50.524+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f75dc113490 con 0x7f75dc10db70 2023-12-11T19:31:51.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.231+0000 7f75d8ff9700 1 -- 172.21.15.66:0/2480447381 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f75d4099df0 con 0x7f75dc10db70 2023-12-11T19:31:51.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.234+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75c40a1480 msgr2=0x7f75c40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.234+0000 7f75e21ff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75c40a1480 0x7f75c40a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f75cc010420 tx=0x7f75cc0073d0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.234+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc10db70 msgr2=0x7f75dc1a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.234+0000 7f75e21ff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc10db70 0x7f75dc1a4ff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f75d400efc0 tx=0x7f75d400c3a0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 shutdown_connections 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75c40a1480 0x7f75c40a3940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75dc074030 0x7f75dc19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75dc10db70 0x7f75dc1a4ff0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 --2- 172.21.15.66:0/2480447381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75dc075610 0x7f75dc19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 >> 172.21.15.66:0/2480447381 conn(0x7f75dc0fcb10 msgr2=0x7f75dc100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 shutdown_connections 2023-12-11T19:31:51.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.235+0000 7f75e21ff700 1 -- 172.21.15.66:0/2480447381 wait complete. 2023-12-11T19:31:51.242 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2023-12-11T19:31:51.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-11T19:31:51.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 -- 172.21.15.66:0/163101847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc0ffe70 msgr2=0x7f95dc100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:51.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 --2- 172.21.15.66:0/163101847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc0ffe70 0x7f95dc100290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009990 tx=0x7f95d8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 -- 172.21.15.66:0/163101847 shutdown_connections 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 --2- 172.21.15.66:0/163101847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc0ffe70 0x7f95dc100290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 --2- 172.21.15.66:0/163101847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc101190 0x7f95dc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 --2- 172.21.15.66:0/163101847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95dc1007d0 0x7f95dc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 -- 172.21.15.66:0/163101847 >> 172.21.15.66:0/163101847 conn(0x7f95dc0fb9f0 msgr2=0x7f95dc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 -- 172.21.15.66:0/163101847 shutdown_connections 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.489+0000 7f95e41ff700 1 -- 172.21.15.66:0/163101847 wait complete. 2023-12-11T19:31:51.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.490+0000 7f95e41ff700 1 Processor -- start 2023-12-11T19:31:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.490+0000 7f95e41ff700 1 -- start start 2023-12-11T19:31:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e41ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95dc0ffe70 0x7f95dc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e41ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 0x7f95dc199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e41ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc101190 0x7f95dc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e41ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc1a1180 con 0x7f95dc1007d0 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e41ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc1a12c0 con 0x7f95dc101190 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e41ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc1a1450 con 0x7f95dc0ffe70 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 0x7f95dc199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 0x7f95dc199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40698/0 (socket says 172.21.15.66:40698) 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 -- 172.21.15.66:0/1830927636 learned_addr learned my addr 172.21.15.66:0/1830927636 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e1f9b700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95dc0ffe70 0x7f95dc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e279c700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc101190 0x7f95dc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 -- 172.21.15.66:0/1830927636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95dc0ffe70 msgr2=0x7f95dc199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95dc0ffe70 0x7f95dc199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 -- 172.21.15.66:0/1830927636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc101190 msgr2=0x7f95dc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc101190 0x7f95dc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.491+0000 7f95e179a700 1 -- 172.21.15.66:0/1830927636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80095f0 con 0x7f95dc1007d0 2023-12-11T19:31:51.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.492+0000 7f95e179a700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 0x7f95dc199b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00d950 tx=0x7f95cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:51.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.492+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00f890 con 0x7f95dc1007d0 2023-12-11T19:31:51.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.492+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95cc00fe80 con 0x7f95dc1007d0 2023-12-11T19:31:51.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.492+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc1a1730 con 0x7f95dc1007d0 2023-12-11T19:31:51.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.492+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00e3b0 con 0x7f95dc1007d0 2023-12-11T19:31:51.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.492+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc1a1c90 con 0x7f95dc1007d0 2023-12-11T19:31:51.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.493+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc10af70 con 0x7f95dc1007d0 2023-12-11T19:31:51.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.495+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f95cc010460 con 0x7f95dc1007d0 2023-12-11T19:31:51.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.498+0000 7f95d2ffd700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f95c80a1500 0x7f95c80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:51.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.498+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f95cc0cc900 con 0x7f95dc1007d0 2023-12-11T19:31:51.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.498+0000 7f95e1f9b700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f95c80a1500 0x7f95c80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:51.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.499+0000 7f95e1f9b700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f95c80a1500 0x7f95c80a39c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005910 tx=0x7f95d800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.501+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f95cc096070 con 0x7f95dc1007d0 2023-12-11T19:31:51.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:51 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/328315380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-11T19:31:51.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:51 smithi167 ceph-mon[102750]: osdmap e83: 12 total, 12 up, 12 in 2023-12-11T19:31:51.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:51 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:51 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/328315380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-11T19:31:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:51 smithi066 ceph-mon[97920]: osdmap e83: 12 total, 12 up, 12 in 2023-12-11T19:31:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:51 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:51 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/328315380' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-11T19:31:51.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:51 smithi070 ceph-mon[104044]: osdmap e83: 12 total, 12 up, 12 in 2023-12-11T19:31:51.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:51 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:51.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:51.762+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f95dc10f0b0 con 0x7f95dc1007d0 2023-12-11T19:31:52.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.477+0000 7f95d2ffd700 1 -- 172.21.15.66:0/1830927636 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f95cc0160c0 con 0x7f95dc1007d0 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f95c80a1500 msgr2=0x7f95c80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f95c80a1500 0x7f95c80a39c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005910 tx=0x7f95d800b5b0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 msgr2=0x7f95dc199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 0x7f95dc199b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00d950 tx=0x7f95cc00dc60 comp rx=0 tx=0).stop 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 shutdown_connections 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f95c80a1500 0x7f95c80a39c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95dc101190 0x7f95dc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95dc0ffe70 0x7f95dc199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 --2- 172.21.15.66:0/1830927636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95dc1007d0 0x7f95dc199b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 >> 172.21.15.66:0/1830927636 conn(0x7f95dc0fb9f0 msgr2=0x7f95dc0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 shutdown_connections 2023-12-11T19:31:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.481+0000 7f95e41ff700 1 -- 172.21.15.66:0/1830927636 wait complete. 2023-12-11T19:31:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: pgmap v303: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2480447381' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2480447381' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: osdmap e84: 12 total, 12 up, 12 in 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T19:31:52.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:52.504 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1830927636' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-11T19:31:52.504 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:52 smithi066 ceph-mon[97920]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-11T19:31:52.505 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:31:52.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: pgmap v303: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:52.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2480447381' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-11T19:31:52.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2480447381' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-11T19:31:52.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: osdmap e84: 12 total, 12 up, 12 in 2023-12-11T19:31:52.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:52.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:52.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:52.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T19:31:52.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:52.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1830927636' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-11T19:31:52.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:52 smithi167 ceph-mon[102750]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: pgmap v303: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2480447381' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2480447381' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: osdmap e84: 12 total, 12 up, 12 in 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:52.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T19:31:52.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:31:52.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1830927636' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-11T19:31:52.611 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:52 smithi070 ceph-mon[104044]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-11T19:31:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 -- 172.21.15.66:0/3080776957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd41048a0 msgr2=0x7f9fd4104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3080776957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd41048a0 0x7f9fd4104d00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00099c0 tx=0x7f9fc0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 -- 172.21.15.66:0/3080776957 shutdown_connections 2023-12-11T19:31:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3080776957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3080776957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd4105240 0x7f9fd410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3080776957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd41048a0 0x7f9fd4104d00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 -- 172.21.15.66:0/3080776957 >> 172.21.15.66:0/3080776957 conn(0x7f9fd40fec40 msgr2=0x7f9fd4101060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 -- 172.21.15.66:0/3080776957 shutdown_connections 2023-12-11T19:31:52.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.721+0000 7f9fdadff700 1 -- 172.21.15.66:0/3080776957 wait complete. 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.722+0000 7f9fdadff700 1 Processor -- start 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 -- start start 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd41048a0 0x7f9fd4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd4105240 0x7f9fd41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a1080 con 0x7f9fd4105240 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a11c0 con 0x7f9fd41036b0 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fdadff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a1350 con 0x7f9fd41048a0 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fd8b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fd8b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:49166/0 (socket says 172.21.15.66:49166) 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fd8b9b700 1 -- 172.21.15.66:0/3637425817 learned_addr learned my addr 172.21.15.66:0/3637425817 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fd939c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd4105240 0x7f9fd41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:52.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.723+0000 7f9fd3fff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd41048a0 0x7f9fd4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd8b9b700 1 -- 172.21.15.66:0/3637425817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd41048a0 msgr2=0x7f9fd4199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd8b9b700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd41048a0 0x7f9fd4199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd8b9b700 1 -- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd4105240 msgr2=0x7f9fd41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd8b9b700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd4105240 0x7f9fd41a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd8b9b700 1 -- 172.21.15.66:0/3637425817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00095f0 con 0x7f9fd41036b0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd8b9b700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4199550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800d980 tx=0x7f9fc800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004020 con 0x7f9fd41036b0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fc800f690 con 0x7f9fd41036b0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd41a1630 con 0x7f9fd41036b0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80039e0 con 0x7f9fd41036b0 2023-12-11T19:31:52.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.724+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd41a1b90 con 0x7f9fd41036b0 2023-12-11T19:31:52.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.725+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd410aba0 con 0x7f9fd41036b0 2023-12-11T19:31:52.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.727+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9fc800f800 con 0x7f9fd41036b0 2023-12-11T19:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.729+0000 7f9fd1ffb700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fc40a1500 0x7f9fc40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.729+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9fc80cc7b0 con 0x7f9fd41036b0 2023-12-11T19:31:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.729+0000 7f9fd3fff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fc40a1500 0x7f9fc40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.730+0000 7f9fd3fff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fc40a1500 0x7f9fc40a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000c00 tx=0x7f9fc0003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.736+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9fc8095ec0 con 0x7f9fd41036b0 2023-12-11T19:31:52.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.998+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd4068230 con 0x7f9fd41036b0 2023-12-11T19:31:53.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:52.999+0000 7f9fd1ffb700 1 -- 172.21.15.66:0/3637425817 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22024 (secure 0 0 0) 0x7f9fc80994e0 con 0x7f9fd41036b0 2023-12-11T19:31:53.001 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:53.001 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":85,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:31:51.763656+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.002+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fc40a1500 msgr2=0x7f9fc40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.002+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fc40a1500 0x7f9fc40a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000c00 tx=0x7f9fc0003680 comp rx=0 tx=0).stop 2023-12-11T19:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 msgr2=0x7f9fd4199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4199550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800d980 tx=0x7f9fc800dc90 comp rx=0 tx=0).stop 2023-12-11T19:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 shutdown_connections 2023-12-11T19:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9fc40a1500 0x7f9fc40a39c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fd41036b0 0x7f9fd4199550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd4105240 0x7f9fd41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 --2- 172.21.15.66:0/3637425817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9fd41048a0 0x7f9fd4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 >> 172.21.15.66:0/3637425817 conn(0x7f9fd40fec40 msgr2=0x7f9fd4107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 shutdown_connections 2023-12-11T19:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.003+0000 7f9fdadff700 1 -- 172.21.15.66:0/3637425817 wait complete. 2023-12-11T19:31:53.025 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:31:53.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 -- 172.21.15.66:0/237675755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 msgr2=0x7f9ee010fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/237675755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee010fe60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b0a0 tx=0x7f9ed800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 -- 172.21.15.66:0/237675755 shutdown_connections 2023-12-11T19:31:53.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/237675755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee010fe60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/237675755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0075610 0x7f9ee0073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/237675755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ee0074030 0x7f9ee00744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 -- 172.21.15.66:0/237675755 >> 172.21.15.66:0/237675755 conn(0x7f9ee00fca10 msgr2=0x7f9ee00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:53.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 -- 172.21.15.66:0/237675755 shutdown_connections 2023-12-11T19:31:53.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.239+0000 7f9ee59ff700 1 -- 172.21.15.66:0/237675755 wait complete. 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 Processor -- start 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 -- start start 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 0x7f9ee019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ee0075610 0x7f9ee019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee01a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a5560 con 0x7f9ee010da70 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a56a0 con 0x7f9ee0075610 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.240+0000 7f9ee59ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a5830 con 0x7f9ee0074030 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee01a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 0x7f9ee019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9ede7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ee0075610 0x7f9ee019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 0x7f9ee019da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:45732/0 (socket says 172.21.15.66:45732) 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 -- 172.21.15.66:0/4130710870 learned_addr learned my addr 172.21.15.66:0/4130710870 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 -- 172.21.15.66:0/4130710870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ee0075610 msgr2=0x7f9ee019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ee0075610 0x7f9ee019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 -- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 msgr2=0x7f9ee01a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee01a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edeffd700 1 -- 172.21.15.66:0/4130710870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009520 con 0x7f9ee0074030 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.241+0000 7f9edf7fe700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee01a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.242+0000 7f9edeffd700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 0x7f9ee019da30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000e9a0 tx=0x7f9ed000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.242+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0009790 con 0x7f9ee0074030 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.242+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ed0010430 con 0x7f9ee0074030 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.242+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed8009d00 con 0x7f9ee0074030 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.242+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000cea0 con 0x7f9ee0074030 2023-12-11T19:31:53.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.242+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee01a5e90 con 0x7f9ee0074030 2023-12-11T19:31:53.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.243+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0103cd0 con 0x7f9ee0074030 2023-12-11T19:31:53.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.245+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9ed00105a0 con 0x7f9ee0074030 2023-12-11T19:31:53.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.247+0000 7f9ee49fd700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ec80a1480 0x7f9ec80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.247+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9ed0014070 con 0x7f9ee0074030 2023-12-11T19:31:53.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.247+0000 7f9ede7fc700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ec80a1480 0x7f9ec80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.247+0000 7f9ede7fc700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ec80a1480 0x7f9ec80a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0075330 tx=0x7f9ed4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:53.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.250+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9ed0096700 con 0x7f9ee0074030 2023-12-11T19:31:53.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:53 smithi066 ceph-mon[97920]: osdmap e85: 12 total, 12 up, 12 in 2023-12-11T19:31:53.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:53 smithi066 ceph-mon[97920]: pgmap v306: 97 pgs: 18 creating+peering, 4 active+clean, 75 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:53.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:53 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1830927636' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-11T19:31:53.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:53 smithi066 ceph-mon[97920]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-11T19:31:53.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:53 smithi066 ceph-mon[97920]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-11T19:31:53.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:53 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3637425817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:53.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.474+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ee0113140 con 0x7f9ee0074030 2023-12-11T19:31:53.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.476+0000 7f9ee49fd700 1 -- 172.21.15.66:0/4130710870 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22011 (secure 0 0 0) 0x7f9ed0099d20 con 0x7f9ee0074030 2023-12-11T19:31:53.477 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:31:53.477 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:31:53.249050+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:31:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.478+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ec80a1480 msgr2=0x7f9ec80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ec80a1480 0x7f9ec80a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0075330 tx=0x7f9ed4009260 comp rx=0 tx=0).stop 2023-12-11T19:31:53.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 msgr2=0x7f9ee019da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 0x7f9ee019da30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000e9a0 tx=0x7f9ed000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 shutdown_connections 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ec80a1480 0x7f9ec80a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee010da70 0x7f9ee01a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ee0074030 0x7f9ee019da30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 --2- 172.21.15.66:0/4130710870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ee0075610 0x7f9ee019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 >> 172.21.15.66:0/4130710870 conn(0x7f9ee00fca10 msgr2=0x7f9ee0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 shutdown_connections 2023-12-11T19:31:53.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.479+0000 7f9ee59ff700 1 -- 172.21.15.66:0/4130710870 wait complete. 2023-12-11T19:31:53.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-11T19:31:53.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:53 smithi167 ceph-mon[102750]: osdmap e85: 12 total, 12 up, 12 in 2023-12-11T19:31:53.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:53 smithi167 ceph-mon[102750]: pgmap v306: 97 pgs: 18 creating+peering, 4 active+clean, 75 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:53.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:53 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1830927636' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-11T19:31:53.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:53 smithi167 ceph-mon[102750]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-11T19:31:53.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:53 smithi167 ceph-mon[102750]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-11T19:31:53.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:53 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3637425817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:53.609 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:53 smithi070 ceph-mon[104044]: osdmap e85: 12 total, 12 up, 12 in 2023-12-11T19:31:53.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:53 smithi070 ceph-mon[104044]: pgmap v306: 97 pgs: 18 creating+peering, 4 active+clean, 75 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T19:31:53.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:53 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1830927636' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-11T19:31:53.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:53 smithi070 ceph-mon[104044]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-11T19:31:53.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:53 smithi070 ceph-mon[104044]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-11T19:31:53.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:53 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3637425817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 -- 172.21.15.66:0/3688139124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 msgr2=0x7f19fc10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3688139124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc10b8e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b0a0 tx=0x7f19f400b3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 -- 172.21.15.66:0/3688139124 shutdown_connections 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3688139124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19fc0755b0 0x7f19fc073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3688139124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc10b8e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3688139124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19fc073fd0 0x7f19fc074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 -- 172.21.15.66:0/3688139124 >> 172.21.15.66:0/3688139124 conn(0x7f19fc0fcb10 msgr2=0x7f19fc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 -- 172.21.15.66:0/3688139124 shutdown_connections 2023-12-11T19:31:53.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.720+0000 7f1a017ff700 1 -- 172.21.15.66:0/3688139124 wait complete. 2023-12-11T19:31:53.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.722+0000 7f1a017ff700 1 Processor -- start 2023-12-11T19:31:53.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.722+0000 7f1a017ff700 1 -- start start 2023-12-11T19:31:53.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.722+0000 7f1a017ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19fc073fd0 0x7f19fc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.722+0000 7f1a017ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19fc0755b0 0x7f19fc199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.722+0000 7f1a017ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f1a017ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc1a1200 con 0x7f19fc109470 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f1a017ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc1a1340 con 0x7f19fc073fd0 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f1a017ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc1a14d0 con 0x7f19fc0755b0 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33868/0 (socket says 172.21.15.66:33868) 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 -- 172.21.15.66:0/3915531927 learned_addr learned my addr 172.21.15.66:0/3915531927 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fa7fc700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19fc0755b0 0x7f19fc199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19faffd700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19fc073fd0 0x7f19fc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 -- 172.21.15.66:0/3915531927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19fc0755b0 msgr2=0x7f19fc199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19fc0755b0 0x7f19fc199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 -- 172.21.15.66:0/3915531927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19fc073fd0 msgr2=0x7f19fc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:53.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19fc073fd0 0x7f19fc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:53.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.723+0000 7f19fb7fe700 1 -- 172.21.15.66:0/3915531927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009d00 con 0x7f19fc109470 2023-12-11T19:31:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.724+0000 7f19fb7fe700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc1a0c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b780 tx=0x7f19f400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.724+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400e040 con 0x7f19fc109470 2023-12-11T19:31:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.724+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19f4003c00 con 0x7f19fc109470 2023-12-11T19:31:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.724+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc1a1750 con 0x7f19fc109470 2023-12-11T19:31:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.724+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc1a1b70 con 0x7f19fc109470 2023-12-11T19:31:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.724+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40052a0 con 0x7f19fc109470 2023-12-11T19:31:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.725+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc108280 con 0x7f19fc109470 2023-12-11T19:31:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.727+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f19f401d480 con 0x7f19fc109470 2023-12-11T19:31:53.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.729+0000 7f19e3fff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19e40a1480 0x7f19e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:53.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.729+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19f40cdc40 con 0x7f19fc109470 2023-12-11T19:31:53.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.729+0000 7f19faffd700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19e40a1480 0x7f19e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:53.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.730+0000 7f19faffd700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19e40a1480 0x7f19e40a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009c60 tx=0x7f19ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:53.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:53.733+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f19f40973e0 con 0x7f19fc109470 2023-12-11T19:31:54.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.048+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f19fc068310 con 0x7f19fc109470 2023-12-11T19:31:54.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.486+0000 7f19e3fff700 1 -- 172.21.15.66:0/3915531927 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f19f409aa00 con 0x7f19fc109470 2023-12-11T19:31:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19e40a1480 msgr2=0x7f19e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19e40a1480 0x7f19e40a3940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009c60 tx=0x7f19ec009260 comp rx=0 tx=0).stop 2023-12-11T19:31:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 msgr2=0x7f19fc1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:54.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc1a0c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b780 tx=0x7f19f400bab0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 shutdown_connections 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19e40a1480 0x7f19e40a3940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19fc073fd0 0x7f19fc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19fc109470 0x7f19fc1a0c30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 --2- 172.21.15.66:0/3915531927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19fc0755b0 0x7f19fc199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 >> 172.21.15.66:0/3915531927 conn(0x7f19fc0fcb10 msgr2=0x7f19fc1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 shutdown_connections 2023-12-11T19:31:54.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.489+0000 7f1a017ff700 1 -- 172.21.15.66:0/3915531927 wait complete. 2023-12-11T19:31:54.509 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-12-11T19:31:54.509 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-12-11T19:31:54.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-11T19:31:54.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: Cluster is now healthy 2023-12-11T19:31:54.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: osdmap e86: 12 total, 12 up, 12 in 2023-12-11T19:31:54.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4130710870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:54.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] up:active 2023-12-11T19:31:54.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:54.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:54.521 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:54 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3915531927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-11T19:31:54.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-11T19:31:54.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: Cluster is now healthy 2023-12-11T19:31:54.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: osdmap e86: 12 total, 12 up, 12 in 2023-12-11T19:31:54.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4130710870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:54.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] up:active 2023-12-11T19:31:54.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:54.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:54.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:54 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3915531927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: Cluster is now healthy 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: osdmap e86: 12 total, 12 up, 12 in 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4130710870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] up:active 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:54.610 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:54 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3915531927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-11T19:31:54.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/3404100010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 msgr2=0x7f9b38073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:54.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/3404100010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 0x7f9b38073a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009990 tx=0x7f9b28009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/3404100010 shutdown_connections 2023-12-11T19:31:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/3404100010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 0x7f9b38073a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/3404100010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 0x7f9b3810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/3404100010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b380744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/3404100010 >> 172.21.15.66:0/3404100010 conn(0x7f9b380fcb10 msgr2=0x7f9b380fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/3404100010 shutdown_connections 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.724+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/3404100010 wait complete. 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.725+0000 7f9b3c9ff700 1 Processor -- start 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 -- start start 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b38199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 0x7f9b38199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 0x7f9b381a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381a11c0 con 0x7f9b38074030 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381a1300 con 0x7f9b38075610 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3c9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381a1490 con 0x7f9b38109460 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b38199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:54.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b38199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33894/0 (socket says 172.21.15.66:33894) 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3659c700 1 -- 172.21.15.66:0/561977836 learned_addr learned my addr 172.21.15.66:0/561977836 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b35d9b700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 0x7f9b38199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b36d9d700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 0x7f9b381a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3659c700 1 -- 172.21.15.66:0/561977836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 msgr2=0x7f9b381a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.726+0000 7f9b3659c700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 0x7f9b381a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b3659c700 1 -- 172.21.15.66:0/561977836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 msgr2=0x7f9b38199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b3659c700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 0x7f9b38199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:54.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b3659c700 1 -- 172.21.15.66:0/561977836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b280095f0 con 0x7f9b38074030 2023-12-11T19:31:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b36d9d700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 0x7f9b381a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:31:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b3659c700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b38199690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b28003ec0 tx=0x7f9b28005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2801d070 con 0x7f9b38074030 2023-12-11T19:31:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b2800bdb0 con 0x7f9b38074030 2023-12-11T19:31:54.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28021410 con 0x7f9b38074030 2023-12-11T19:31:54.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b381a1710 con 0x7f9b38074030 2023-12-11T19:31:54.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.727+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b381a1ae0 con 0x7f9b38074030 2023-12-11T19:31:54.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.728+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b38108220 con 0x7f9b38074030 2023-12-11T19:31:54.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.730+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9b2800f660 con 0x7f9b38074030 2023-12-11T19:31:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.732+0000 7f9b277fe700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b200a1480 0x7f9b200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.732+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b280ce390 con 0x7f9b38074030 2023-12-11T19:31:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.732+0000 7f9b35d9b700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b200a1480 0x7f9b200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.732+0000 7f9b35d9b700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b200a1480 0x7f9b200a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b38075330 tx=0x7f9b2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:54.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:54.733+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b28097b30 con 0x7f9b38074030 2023-12-11T19:31:55.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.002+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f9b3810ee90 con 0x7f9b38074030 2023-12-11T19:31:55.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.492+0000 7f9b277fe700 1 -- 172.21.15.66:0/561977836 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f9b2809b150 con 0x7f9b38074030 2023-12-11T19:31:55.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b200a1480 msgr2=0x7f9b200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:55.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b200a1480 0x7f9b200a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b38075330 tx=0x7f9b2c009260 comp rx=0 tx=0).stop 2023-12-11T19:31:55.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 msgr2=0x7f9b38199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:55.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b38199690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b28003ec0 tx=0x7f9b28005740 comp rx=0 tx=0).stop 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 shutdown_connections 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9b200a1480 0x7f9b200a3940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b38109460 0x7f9b381a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38074030 0x7f9b38199690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 --2- 172.21.15.66:0/561977836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b38075610 0x7f9b38199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 >> 172.21.15.66:0/561977836 conn(0x7f9b380fcb10 msgr2=0x7f9b38105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 shutdown_connections 2023-12-11T19:31:55.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.495+0000 7f9b3c9ff700 1 -- 172.21.15.66:0/561977836 wait complete. 2023-12-11T19:31:55.521 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-11T19:31:55.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 -- 172.21.15.66:0/3282339408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358102000 msgr2=0x7f23581043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:55.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 --2- 172.21.15.66:0/3282339408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358102000 0x7f23581043f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2354009990 tx=0x7f2354009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 -- 172.21.15.66:0/3282339408 shutdown_connections 2023-12-11T19:31:55.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 --2- 172.21.15.66:0/3282339408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358107260 0x7f23581096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 --2- 172.21.15.66:0/3282339408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358102000 0x7f23581043f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 --2- 172.21.15.66:0/3282339408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358104930 0x7f2358106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 -- 172.21.15.66:0/3282339408 >> 172.21.15.66:0/3282339408 conn(0x7f23580fb9f0 msgr2=0x7f23580fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.733+0000 7f23605ff700 1 -- 172.21.15.66:0/3282339408 shutdown_connections 2023-12-11T19:31:55.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.734+0000 7f23605ff700 1 -- 172.21.15.66:0/3282339408 wait complete. 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.734+0000 7f23605ff700 1 Processor -- start 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 -- start start 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358102000 0x7f2358199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 0x7f2358199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358107260 0x7f23581a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581a1170 con 0x7f2358104930 2023-12-11T19:31:55.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581a12b0 con 0x7f2358107260 2023-12-11T19:31:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f23605ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581a1440 con 0x7f2358102000 2023-12-11T19:31:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f235db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 0x7f2358199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f235db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 0x7f2358199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33900/0 (socket says 172.21.15.66:33900) 2023-12-11T19:31:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f235db9a700 1 -- 172.21.15.66:0/1291098936 learned_addr learned my addr 172.21.15.66:0/1291098936 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f235e39b700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358102000 0x7f2358199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:55.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.735+0000 7f235eb9c700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358107260 0x7f23581a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235db9a700 1 -- 172.21.15.66:0/1291098936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358102000 msgr2=0x7f2358199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235db9a700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358102000 0x7f2358199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235db9a700 1 -- 172.21.15.66:0/1291098936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358107260 msgr2=0x7f23581a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235db9a700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358107260 0x7f23581a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235db9a700 1 -- 172.21.15.66:0/1291098936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23540095f0 con 0x7f2358104930 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235e39b700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358102000 0x7f2358199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235eb9c700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358107260 0x7f23581a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f235db9a700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 0x7f2358199b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f234800d950 tx=0x7f234800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234800f890 con 0x7f2358104930 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f234800fe80 con 0x7f2358104930 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23581a1720 con 0x7f2358104930 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234800e3b0 con 0x7f2358104930 2023-12-11T19:31:55.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.736+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23581a1c80 con 0x7f2358104930 2023-12-11T19:31:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.738+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2358068310 con 0x7f2358104930 2023-12-11T19:31:55.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.740+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2348010460 con 0x7f2358104930 2023-12-11T19:31:55.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.741+0000 7f234f7fe700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f23440a1500 0x7f23440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.741+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f23480cc990 con 0x7f2358104930 2023-12-11T19:31:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.741+0000 7f235e39b700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f23440a1500 0x7f23440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:55.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.742+0000 7f235e39b700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f23440a1500 0x7f23440a39c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2354006010 tx=0x7f235400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:55.745+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f23480960a0 con 0x7f2358104930 2023-12-11T19:31:55.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: pgmap v308: 97 pgs: 18 creating+peering, 57 active+clean, 22 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2023-12-11T19:31:55.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3915531927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-11T19:31:55.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/561977836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:55.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:55 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: pgmap v308: 97 pgs: 18 creating+peering, 57 active+clean, 22 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3915531927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/561977836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:55.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:55 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: pgmap v308: 97 pgs: 18 creating+peering, 57 active+clean, 22 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2023-12-11T19:31:55.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3915531927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-11T19:31:55.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/561977836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:31:55.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:31:56.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.014+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f23581a1f40 con 0x7f2358104930 2023-12-11T19:31:56.360 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:31:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b[140194]: 2023-12-11T19:31:55.941+0000 7f664657e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-11T19:31:56.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.476+0000 7f234f7fe700 1 -- 172.21.15.66:0/1291098936 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f23480160c0 con 0x7f2358104930 2023-12-11T19:31:56.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f23440a1500 msgr2=0x7f23440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:56.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f23440a1500 0x7f23440a39c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2354006010 tx=0x7f235400b610 comp rx=0 tx=0).stop 2023-12-11T19:31:56.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 msgr2=0x7f2358199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:56.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 0x7f2358199b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f234800d950 tx=0x7f234800dc60 comp rx=0 tx=0).stop 2023-12-11T19:31:56.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 shutdown_connections 2023-12-11T19:31:56.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f23440a1500 0x7f23440a39c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2358107260 0x7f23581a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358102000 0x7f2358199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 --2- 172.21.15.66:0/1291098936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2358104930 0x7f2358199b80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.479+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 >> 172.21.15.66:0/1291098936 conn(0x7f23580fb9f0 msgr2=0x7f23580fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.480+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 shutdown_connections 2023-12-11T19:31:56.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.480+0000 7f23605ff700 1 -- 172.21.15.66:0/1291098936 wait complete. 2023-12-11T19:31:56.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-11T19:31:56.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/3494480480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f041015a0 msgr2=0x7f0f041019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:56.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/3494480480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f041015a0 0x7f0f041019c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40098f0 tx=0x7f0ef4009c00 comp rx=0 tx=0).stop 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/3494480480 shutdown_connections 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/3494480480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04103130 0x7f0f041099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/3494480480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f041015a0 0x7f0f041019c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/3494480480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f04102790 0x7f0f04102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/3494480480 >> 172.21.15.66:0/3494480480 conn(0x7f0f040fcb10 msgr2=0x7f0f040fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.721+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/3494480480 shutdown_connections 2023-12-11T19:31:56.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.722+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/3494480480 wait complete. 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.722+0000 7f0f0a7ff700 1 Processor -- start 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 -- start start 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f041015a0 0x7f0f04197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 0x7f0f04197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f04103130 0x7f0f0419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419f020 con 0x7f0f04102790 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419f160 con 0x7f0f041015a0 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f0a7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419f2f0 con 0x7f0f04103130 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0efbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 0x7f0f04197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:56.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0efbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 0x7f0f04197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33926/0 (socket says 172.21.15.66:33926) 2023-12-11T19:31:56.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0efbfff700 1 -- 172.21.15.66:0/1135680624 learned_addr learned my addr 172.21.15.66:0/1135680624 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:56.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f03fff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f041015a0 0x7f0f04197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.723+0000 7f0f08d9c700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f04103130 0x7f0f0419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0efbfff700 1 -- 172.21.15.66:0/1135680624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f04103130 msgr2=0x7f0f0419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0efbfff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f04103130 0x7f0f0419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0efbfff700 1 -- 172.21.15.66:0/1135680624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f041015a0 msgr2=0x7f0f04197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0efbfff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f041015a0 0x7f0f04197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0efbfff700 1 -- 172.21.15.66:0/1135680624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc009d00 con 0x7f0f04102790 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0efbfff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 0x7f0f04197a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00d950 tx=0x7f0eec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00f890 con 0x7f0f04102790 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0eec00fe80 con 0x7f0f04102790 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4009520 con 0x7f0f04102790 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00e3b0 con 0x7f0f04102790 2023-12-11T19:31:56.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.724+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0419f950 con 0x7f0f04102790 2023-12-11T19:31:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.725+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f04108c00 con 0x7f0f04102790 2023-12-11T19:31:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.727+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0eec010460 con 0x7f0f04102790 2023-12-11T19:31:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.729+0000 7f0f01ffb700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0ef00a1480 0x7f0ef00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.729+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0eec0ccdb0 con 0x7f0f04102790 2023-12-11T19:31:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.729+0000 7f0f03fff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0ef00a1480 0x7f0ef00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:56.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.730+0000 7f0f03fff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0ef00a1480 0x7f0ef00a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009fd0 tx=0x7f0ef4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:56.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:56.733+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0eec0d1070 con 0x7f0f04102790 2023-12-11T19:31:56.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-11T19:31:56.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/561977836' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-11T19:31:56.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-11T19:31:56.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1291098936' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1291098936' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] up:active 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-12-11T19:31:56.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:56 smithi167 ceph-mon[102750]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-12-11T19:31:56.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/561977836' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1291098936' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1291098936' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] up:active 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-12-11T19:31:56.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:56 smithi066 ceph-mon[97920]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-12-11T19:31:56.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-11T19:31:56.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/561977836' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-11T19:31:56.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-11T19:31:56.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1291098936' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1291098936' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] up:active 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-12-11T19:31:56.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:56 smithi070 ceph-mon[104044]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-12-11T19:31:57.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.002+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f0f04068310 con 0x7f0f04102790 2023-12-11T19:31:57.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.507+0000 7f0f01ffb700 1 -- 172.21.15.66:0/1135680624 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0eec0200b0 con 0x7f0f04102790 2023-12-11T19:31:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0ef00a1480 msgr2=0x7f0ef00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0ef00a1480 0x7f0ef00a3940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009fd0 tx=0x7f0ef4019040 comp rx=0 tx=0).stop 2023-12-11T19:31:57.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 msgr2=0x7f0f04197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 0x7f0f04197a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00d950 tx=0x7f0eec00dc60 comp rx=0 tx=0).stop 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 shutdown_connections 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0ef00a1480 0x7f0ef00a3940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f041015a0 0x7f0f04197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f04103130 0x7f0f0419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 --2- 172.21.15.66:0/1135680624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f04102790 0x7f0f04197a80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 >> 172.21.15.66:0/1135680624 conn(0x7f0f040fcb10 msgr2=0x7f0f04105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 shutdown_connections 2023-12-11T19:31:57.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.510+0000 7f0f0a7ff700 1 -- 172.21.15.66:0/1135680624 wait complete. 2023-12-11T19:31:57.535 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-12-11T19:31:57.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-11T19:31:57.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 -- 172.21.15.66:0/3599117322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1037b0 msgr2=0x7f3d5c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:57.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 --2- 172.21.15.66:0/3599117322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1037b0 0x7f3d5c103bd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0099c0 tx=0x7f3d4c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 -- 172.21.15.66:0/3599117322 shutdown_connections 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 --2- 172.21.15.66:0/3599117322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c105340 0x7f3d5c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 --2- 172.21.15.66:0/3599117322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1037b0 0x7f3d5c103bd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 --2- 172.21.15.66:0/3599117322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 -- 172.21.15.66:0/3599117322 >> 172.21.15.66:0/3599117322 conn(0x7f3d5c0fed20 msgr2=0x7f3d5c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 -- 172.21.15.66:0/3599117322 shutdown_connections 2023-12-11T19:31:57.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.747+0000 7f3d609ff700 1 -- 172.21.15.66:0/3599117322 wait complete. 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.748+0000 7f3d609ff700 1 Processor -- start 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.748+0000 7f3d609ff700 1 -- start start 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d609ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c1037b0 0x7f3d5c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d609ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d609ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d5c105340 0x7f3d5c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d609ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1a11a0 con 0x7f3d5c1049a0 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d609ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1a12e0 con 0x7f3d5c1037b0 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d609ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1a1470 con 0x7f3d5c105340 2023-12-11T19:31:57.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33934/0 (socket says 172.21.15.66:33934) 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 -- 172.21.15.66:0/946437242 learned_addr learned my addr 172.21.15.66:0/946437242 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d5a59c700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c1037b0 0x7f3d5c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 -- 172.21.15.66:0/946437242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d5c105340 msgr2=0x7f3d5c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d5c105340 0x7f3d5c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 -- 172.21.15.66:0/946437242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c1037b0 msgr2=0x7f3d5c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c1037b0 0x7f3d5c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.749+0000 7f3d53fff700 1 -- 172.21.15.66:0/946437242 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0095f0 con 0x7f3d5c1049a0 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.750+0000 7f3d53fff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c199bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400d950 tx=0x7f3d4400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.750+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44009ac0 con 0x7f3d5c1049a0 2023-12-11T19:31:57.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.750+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d44009c20 con 0x7f3d5c1049a0 2023-12-11T19:31:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.750+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400f3c0 con 0x7f3d5c1049a0 2023-12-11T19:31:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.750+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c1a1750 con 0x7f3d5c1049a0 2023-12-11T19:31:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.750+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c1a1c10 con 0x7f3d5c1049a0 2023-12-11T19:31:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.751+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c10acd0 con 0x7f3d5c1049a0 2023-12-11T19:31:57.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.753+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3d44010460 con 0x7f3d5c1049a0 2023-12-11T19:31:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.754+0000 7f3d537fe700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d480a1500 0x7f3d480a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.755+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3d440cca90 con 0x7f3d5c1049a0 2023-12-11T19:31:57.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.755+0000 7f3d5a59c700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d480a1500 0x7f3d480a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:57.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.755+0000 7f3d5a59c700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d480a1500 0x7f3d480a39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:57.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:57.757+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3d440961a0 con 0x7f3d5c1049a0 2023-12-11T19:31:57.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:57 smithi167 ceph-mon[102750]: pgmap v309: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-12-11T19:31:57.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:57 smithi167 ceph-mon[102750]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-12-11T19:31:57.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:57 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1135680624' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-11T19:31:57.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:57 smithi066 ceph-mon[97920]: pgmap v309: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-12-11T19:31:57.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:57 smithi066 ceph-mon[97920]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-12-11T19:31:57.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:57 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1135680624' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-11T19:31:57.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:57 smithi070 ceph-mon[104044]: pgmap v309: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-12-11T19:31:57.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:57 smithi070 ceph-mon[104044]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-12-11T19:31:57.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:57 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1135680624' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-11T19:31:58.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.004+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d5c02d380 con 0x7f3d480a1500 2023-12-11T19:31:58.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.056+0000 7f3d537fe700 1 -- 172.21.15.66:0/946437242 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d5c02d380 con 0x7f3d480a1500 2023-12-11T19:31:58.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d480a1500 msgr2=0x7f3d480a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d480a1500 0x7f3d480a39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c01a040 comp rx=0 tx=0).stop 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 msgr2=0x7f3d5c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c199bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400d950 tx=0x7f3d4400dd10 comp rx=0 tx=0).stop 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 shutdown_connections 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d480a1500 0x7f3d480a39c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c1037b0 0x7f3d5c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d5c105340 0x7f3d5c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 --2- 172.21.15.66:0/946437242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d5c1049a0 0x7f3d5c199bb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 >> 172.21.15.66:0/946437242 conn(0x7f3d5c0fed20 msgr2=0x7f3d5c107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 shutdown_connections 2023-12-11T19:31:58.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.059+0000 7f3d609ff700 1 -- 172.21.15.66:0/946437242 wait complete. 2023-12-11T19:31:58.079 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-11T19:31:58.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 -- 172.21.15.66:0/4181111345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b0075610 msgr2=0x7fd7b0073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/4181111345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b0075610 0x7fd7b0073a60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009990 tx=0x7fd7a4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 -- 172.21.15.66:0/4181111345 shutdown_connections 2023-12-11T19:31:58.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/4181111345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7b010db70 0x7fd7b010ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/4181111345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b0075610 0x7fd7b0073a60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/4181111345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0074030 0x7fd7b00744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 -- 172.21.15.66:0/4181111345 >> 172.21.15.66:0/4181111345 conn(0x7fd7b00fcb10 msgr2=0x7fd7b00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:58.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.290+0000 7fd7b6bff700 1 -- 172.21.15.66:0/4181111345 shutdown_connections 2023-12-11T19:31:58.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.291+0000 7fd7b6bff700 1 -- 172.21.15.66:0/4181111345 wait complete. 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.291+0000 7fd7b6bff700 1 Processor -- start 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 -- start start 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7b0074030 0x7fd7b019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0075610 0x7fd7b019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 0x7fd7b01a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01a5590 con 0x7fd7b010db70 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01a56d0 con 0x7fd7b0074030 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b6bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01a5860 con 0x7fd7b0075610 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b519c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 0x7fd7b01a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b519c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 0x7fd7b01a4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33968/0 (socket says 172.21.15.66:33968) 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b519c700 1 -- 172.21.15.66:0/1942978487 learned_addr learned my addr 172.21.15.66:0/1942978487 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7b499b700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7b0074030 0x7fd7b019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.292+0000 7fd7affff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0075610 0x7fd7b019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b519c700 1 -- 172.21.15.66:0/1942978487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0075610 msgr2=0x7fd7b019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b519c700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0075610 0x7fd7b019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b519c700 1 -- 172.21.15.66:0/1942978487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7b0074030 msgr2=0x7fd7b019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b519c700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7b0074030 0x7fd7b019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b519c700 1 -- 172.21.15.66:0/1942978487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40095f0 con 0x7fd7b010db70 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b519c700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 0x7fd7b01a4fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8007ee0 tx=0x7fd7a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800dd20 con 0x7fd7b010db70 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a800f040 con 0x7fd7b010db70 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b01a5b40 con 0x7fd7b010db70 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004750 con 0x7fd7b010db70 2023-12-11T19:31:58.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.293+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b01a6000 con 0x7fd7b010db70 2023-12-11T19:31:58.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.294+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b0103dd0 con 0x7fd7b010db70 2023-12-11T19:31:58.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.296+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd7a8003a00 con 0x7fd7b010db70 2023-12-11T19:31:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.298+0000 7fd7adffb700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd7a00a1480 0x7fd7a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.298+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd7a80cc510 con 0x7fd7b010db70 2023-12-11T19:31:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.298+0000 7fd7b499b700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd7a00a1480 0x7fd7a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.299+0000 7fd7b499b700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd7a00a1480 0x7fd7a00a3940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4006010 tx=0x7fd7a400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.301+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd7a804f2f0 con 0x7fd7b010db70 2023-12-11T19:31:58.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:31:58.011+0000 7f9f3d5c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T19:31:58.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:31:58.011+0000 7f9f3d5c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T19:31:58.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:31:58.011+0000 7f9f3d5c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T19:31:58.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:31:58.011+0000 7f9f3d5c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T19:31:58.313 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:31:58.011+0000 7f9f3d5c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T19:31:58.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.533+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fd7a8095ff0 con 0x7fd7b010db70 2023-12-11T19:31:58.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.563+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b0061a10 con 0x7fd7a00a1480 2023-12-11T19:31:58.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.567+0000 7fd7adffb700 1 -- 172.21.15.66:0/1942978487 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fd7b0061a10 con 0x7fd7a00a1480 2023-12-11T19:31:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106 2023-12-11T19:31:58.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd7a00a1480 msgr2=0x7fd7a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd7a00a1480 0x7fd7a00a3940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4006010 tx=0x7fd7a400b610 comp rx=0 tx=0).stop 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 msgr2=0x7fd7b01a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 0x7fd7b01a4fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8007ee0 tx=0x7fd7a800d3b0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 shutdown_connections 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd7a00a1480 0x7fd7a00a3940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7b0074030 0x7fd7b019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd7b010db70 0x7fd7b01a4fc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 --2- 172.21.15.66:0/1942978487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0075610 0x7fd7b019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 >> 172.21.15.66:0/1942978487 conn(0x7fd7b00fcb10 msgr2=0x7fd7b0100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 shutdown_connections 2023-12-11T19:31:58.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.570+0000 7fd7b6bff700 1 -- 172.21.15.66:0/1942978487 wait complete. 2023-12-11T19:31:58.590 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-11T19:31:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1135680624' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-11T19:31:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] up:active 2023-12-11T19:31:58.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-11T19:31:58.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-12-11T19:31:58.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-12-11T19:31:58.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-12-11T19:31:58.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:58 smithi167 ceph-mon[102750]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/1235925560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8103130 msgr2=0x7f7ec81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/1235925560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8103130 0x7f7ec81099e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000b0d0 tx=0x7f7ec000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/1235925560 shutdown_connections 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/1235925560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ec81015a0 0x7f7ec81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/1235925560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec8103130 0x7f7ec81099e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/1235925560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 0x7f7ec8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/1235925560 >> 172.21.15.66:0/1235925560 conn(0x7f7ec80fcb10 msgr2=0x7f7ec80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/1235925560 shutdown_connections 2023-12-11T19:31:58.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.802+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/1235925560 wait complete. 2023-12-11T19:31:58.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.803+0000 7f7ecf5ff700 1 Processor -- start 2023-12-11T19:31:58.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.803+0000 7f7ecf5ff700 1 -- start start 2023-12-11T19:31:58.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.803+0000 7f7ecf5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 0x7f7ec8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecf5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 0x7f7ec8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecf5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ec8103130 0x7f7ec81a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecf5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81a12a0 con 0x7f7ec81015a0 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecf5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81a13e0 con 0x7f7ec8103130 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecf5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81a1570 con 0x7f7ec8102790 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 0x7f7ec8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 0x7f7ec8199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33984/0 (socket says 172.21.15.66:33984) 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 -- 172.21.15.66:0/2469622697 learned_addr learned my addr 172.21.15.66:0/2469622697 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecdb9c700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ec8103130 0x7f7ec81a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 -- 172.21.15.66:0/2469622697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 msgr2=0x7f7ec8199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7eccb9a700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 0x7f7ec8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 0x7f7ec8199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 -- 172.21.15.66:0/2469622697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ec8103130 msgr2=0x7f7ec81a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ec8103130 0x7f7ec81a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7ecd39b700 1 -- 172.21.15.66:0/2469622697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0009d00 con 0x7f7ec81015a0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.804+0000 7f7eccb9a700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 0x7f7ec8199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.805+0000 7f7ecd39b700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 0x7f7ec8199770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400e970 tx=0x7f7ec400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.805+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4009790 con 0x7f7ec81015a0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.805+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ec400eec0 con 0x7f7ec81015a0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.805+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec81a1850 con 0x7f7ec81015a0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.805+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4003c10 con 0x7f7ec81015a0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.805+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec81a1d10 con 0x7f7ec81015a0 2023-12-11T19:31:58.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.806+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec8068760 con 0x7f7ec81015a0 2023-12-11T19:31:58.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.808+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7ec4003db0 con 0x7f7ec81015a0 2023-12-11T19:31:58.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.810+0000 7f7ebe7fc700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7eb40a1510 0x7f7eb40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:58.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.810+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7ec4014070 con 0x7f7ec81015a0 2023-12-11T19:31:58.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.810+0000 7f7eccb9a700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7eb40a1510 0x7f7eb40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:58.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.811+0000 7f7eccb9a700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7eb40a1510 0x7f7eb40a39d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009be0 tx=0x7f7eb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1135680624' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] up:active 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-12-11T19:31:58.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:58 smithi066 ceph-mon[97920]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:58.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:58.815+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7ec40965f0 con 0x7f7ec81015a0 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1135680624' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] up:active 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-12-11T19:31:58.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:58 smithi070 ceph-mon[104044]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:59.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.062+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec8074f20 con 0x7f7eb40a1510 2023-12-11T19:31:59.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.089+0000 7f7ebe7fc700 1 -- 172.21.15.66:0/2469622697 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7ec8074f20 con 0x7f7eb40a1510 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7eb40a1510 msgr2=0x7f7eb40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7eb40a1510 0x7f7eb40a39d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009be0 tx=0x7f7eb8009260 comp rx=0 tx=0).stop 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 msgr2=0x7f7ec8199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 0x7f7ec8199770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400e970 tx=0x7f7ec400ec80 comp rx=0 tx=0).stop 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 shutdown_connections 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7eb40a1510 0x7f7eb40a39d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ec8103130 0x7f7ec81a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ec81015a0 0x7f7ec8199770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 --2- 172.21.15.66:0/2469622697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ec8102790 0x7f7ec8199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 >> 172.21.15.66:0/2469622697 conn(0x7f7ec80fcb10 msgr2=0x7f7ec8073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:59.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 shutdown_connections 2023-12-11T19:31:59.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.091+0000 7f7ecf5ff700 1 -- 172.21.15.66:0/2469622697 wait complete. 2023-12-11T19:31:59.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 -- 172.21.15.66:0/1026385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27181015a0 msgr2=0x7f27181019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 --2- 172.21.15.66:0/1026385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27181015a0 0x7f27181019c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27080099c0 tx=0x7f2708009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 -- 172.21.15.66:0/1026385760 shutdown_connections 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 --2- 172.21.15.66:0/1026385760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2718103130 0x7f27181099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 --2- 172.21.15.66:0/1026385760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27181015a0 0x7f27181019c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 --2- 172.21.15.66:0/1026385760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2718102790 0x7f2718102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 -- 172.21.15.66:0/1026385760 >> 172.21.15.66:0/1026385760 conn(0x7f27180fcb10 msgr2=0x7f27180fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 -- 172.21.15.66:0/1026385760 shutdown_connections 2023-12-11T19:31:59.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.323+0000 7f271cbff700 1 -- 172.21.15.66:0/1026385760 wait complete. 2023-12-11T19:31:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.324+0000 7f271cbff700 1 Processor -- start 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.324+0000 7f271cbff700 1 -- start start 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.324+0000 7f271cbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27181015a0 0x7f2718075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.324+0000 7f271cbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 0x7f27180739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.324+0000 7f271cbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2718103130 0x7f2718073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f271cbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27180746a0 con 0x7f2718102790 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f271cbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2718074810 con 0x7f2718103130 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f271cbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27181a14b0 con 0x7f27181015a0 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 0x7f27180739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 0x7f27180739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33998/0 (socket says 172.21.15.66:33998) 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 -- 172.21.15.66:0/2320884693 learned_addr learned my addr 172.21.15.66:0/2320884693 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f271659c700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27181015a0 0x7f2718075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f2716d9d700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2718103130 0x7f2718073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 -- 172.21.15.66:0/2320884693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27181015a0 msgr2=0x7f2718075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27181015a0 0x7f2718075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 -- 172.21.15.66:0/2320884693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2718103130 msgr2=0x7f2718073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2718103130 0x7f2718073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 -- 172.21.15.66:0/2320884693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27080095f0 con 0x7f2718102790 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.325+0000 7f270ffff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 0x7f27180739c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f270000d950 tx=0x7f270000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.326+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000098e0 con 0x7f2718102790 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.326+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27181a1740 con 0x7f2718102790 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.326+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2700010460 con 0x7f2718102790 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.326+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000f620 con 0x7f2718102790 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.326+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27181a1c50 con 0x7f2718102790 2023-12-11T19:31:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.327+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2718108a70 con 0x7f2718102790 2023-12-11T19:31:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.329+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f27000105d0 con 0x7f2718102790 2023-12-11T19:31:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.330+0000 7f270f7fe700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27040a1510 0x7f27040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.330+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f27000ccc40 con 0x7f2718102790 2023-12-11T19:31:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.331+0000 7f271659c700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27040a1510 0x7f27040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.331+0000 7f271659c700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27040a1510 0x7f27040a39d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2708006010 tx=0x7f2708003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.334+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f27000961f0 con 0x7f2718102790 2023-12-11T19:31:59.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.587+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2718061a10 con 0x7f27040a1510 2023-12-11T19:31:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.591+0000 7f270f7fe700 1 -- 172.21.15.66:0/2320884693 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2718061a10 con 0x7f27040a1510 2023-12-11T19:31:59.592 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv_1/aaff15a4-bd47-4f64-9d00-854111a3ead1 2023-12-11T19:31:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27040a1510 msgr2=0x7f27040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27040a1510 0x7f27040a39d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2708006010 tx=0x7f2708003740 comp rx=0 tx=0).stop 2023-12-11T19:31:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 msgr2=0x7f27180739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 0x7f27180739c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f270000d950 tx=0x7f270000dc60 comp rx=0 tx=0).stop 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 shutdown_connections 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27040a1510 0x7f27040a39d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2718103130 0x7f2718073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27181015a0 0x7f2718075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 --2- 172.21.15.66:0/2320884693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2718102790 0x7f27180739c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 >> 172.21.15.66:0/2320884693 conn(0x7f27180fcb10 msgr2=0x7f2718105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 shutdown_connections 2023-12-11T19:31:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.594+0000 7f271cbff700 1 -- 172.21.15.66:0/2320884693 wait complete. 2023-12-11T19:31:59.614 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:31:59.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: pgmap v310: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 290 B/s rd, 3.4 KiB/s wr, 10 op/s 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] up:active 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: Cluster is now healthy 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: mgrmap e17: x(active, since 7m), standbys: y, z 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:59.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:31:59 smithi066 ceph-mon[97920]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 -- 172.21.15.66:0/1680200919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 msgr2=0x7f16c01018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 --2- 172.21.15.66:0/1680200919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c01018c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f16b00099c0 tx=0x7f16b0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 -- 172.21.15.66:0/1680200919 shutdown_connections 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 --2- 172.21.15.66:0/1680200919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0103030 0x7f16c01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 --2- 172.21.15.66:0/1680200919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c01018c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 --2- 172.21.15.66:0/1680200919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0102690 0x7f16c0102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 -- 172.21.15.66:0/1680200919 >> 172.21.15.66:0/1680200919 conn(0x7f16c00fca10 msgr2=0x7f16c00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 -- 172.21.15.66:0/1680200919 shutdown_connections 2023-12-11T19:31:59.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.822+0000 7f16c61ff700 1 -- 172.21.15.66:0/1680200919 wait complete. 2023-12-11T19:31:59.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.823+0000 7f16c61ff700 1 Processor -- start 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.823+0000 7f16c61ff700 1 -- start start 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16c61ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16c61ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0102690 0x7f16c0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16c61ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0103030 0x7f16c01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16c61ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01a1090 con 0x7f16c01014a0 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16c61ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01a11d0 con 0x7f16c0103030 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16c61ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01a1360 con 0x7f16c0102690 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c0199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34022/0 (socket says 172.21.15.66:34022) 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 -- 172.21.15.66:0/2523103056 learned_addr learned my addr 172.21.15.66:0/2523103056 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bffff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0103030 0x7f16c01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16beffd700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0102690 0x7f16c0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 -- 172.21.15.66:0/2523103056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0102690 msgr2=0x7f16c0199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0102690 0x7f16c0199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 -- 172.21.15.66:0/2523103056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0103030 msgr2=0x7f16c01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0103030 0x7f16c01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16bf7fe700 1 -- 172.21.15.66:0/2523103056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b00095f0 con 0x7f16c01014a0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.824+0000 7f16beffd700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0102690 0x7f16c0199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.825+0000 7f16bf7fe700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c0199600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009f50 tx=0x7f16b0003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.825+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b001d070 con 0x7f16c01014a0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.825+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16b000bda0 con 0x7f16c01014a0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.825+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c01a15e0 con 0x7f16c01014a0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.825+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0022dc0 con 0x7f16c01014a0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.825+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c01a1a60 con 0x7f16c01014a0 2023-12-11T19:31:59.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.826+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0068230 con 0x7f16c01014a0 2023-12-11T19:31:59.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.828+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f16b0004000 con 0x7f16c01014a0 2023-12-11T19:31:59.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.829+0000 7f16bcff9700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f16a80a1590 0x7f16a80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:31:59.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.829+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f16b00ce1f0 con 0x7f16c01014a0 2023-12-11T19:31:59.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.830+0000 7f16beffd700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f16a80a1590 0x7f16a80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:31:59.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.830+0000 7f16beffd700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f16a80a1590 0x7f16a80a3a50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005d90 tx=0x7f16b4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:31:59.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:31:59.833+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f16b0097830 con 0x7f16c01014a0 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: pgmap v310: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 290 B/s rd, 3.4 KiB/s wr, 10 op/s 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] up:active 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: Cluster is now healthy 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-12-11T19:31:59.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: mgrmap e17: x(active, since 7m), standbys: y, z 2023-12-11T19:31:59.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-12-11T19:31:59.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-12-11T19:31:59.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:31:59.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:31:59 smithi070 ceph-mon[104044]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: pgmap v310: 97 pgs: 97 active+clean; 580 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 290 B/s rd, 3.4 KiB/s wr, 10 op/s 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] up:active 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: Cluster is now healthy 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: mgrmap e17: x(active, since 7m), standbys: y, z 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-12-11T19:32:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:32:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:31:59 smithi167 ceph-mon[102750]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:32:00.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.094+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16c010ce00 con 0x7f16c01014a0 2023-12-11T19:32:00.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.094+0000 7f16bcff9700 1 -- 172.21.15.66:0/2523103056 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f16b009ae50 con 0x7f16c01014a0 2023-12-11T19:32:00.095 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:00.095 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f16a80a1590 msgr2=0x7f16a80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f16a80a1590 0x7f16a80a3a50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005d90 tx=0x7f16b4005d20 comp rx=0 tx=0).stop 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 msgr2=0x7f16c0199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c0199600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009f50 tx=0x7f16b0003b00 comp rx=0 tx=0).stop 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 shutdown_connections 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f16a80a1590 0x7f16a80a3a50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0103030 0x7f16c01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16c01014a0 0x7f16c0199600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 --2- 172.21.15.66:0/2523103056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16c0102690 0x7f16c0199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 >> 172.21.15.66:0/2523103056 conn(0x7f16c00fca10 msgr2=0x7f16c0107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 shutdown_connections 2023-12-11T19:32:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.097+0000 7f16c61ff700 1 -- 172.21.15.66:0/2523103056 wait complete. 2023-12-11T19:32:00.099 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:32:00.118 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 -- 172.21.15.66:0/2841628146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4810103030 msgr2=0x7f48101098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 --2- 172.21.15.66:0/2841628146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4810103030 0x7f48101098e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f480800b0d0 tx=0x7f480800b3e0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 -- 172.21.15.66:0/2841628146 shutdown_connections 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 --2- 172.21.15.66:0/2841628146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4810103030 0x7f48101098e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 --2- 172.21.15.66:0/2841628146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48101014a0 0x7f48101018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 --2- 172.21.15.66:0/2841628146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4810102690 0x7f4810102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 -- 172.21.15.66:0/2841628146 >> 172.21.15.66:0/2841628146 conn(0x7f48100fca10 msgr2=0x7f48100fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:00.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 -- 172.21.15.66:0/2841628146 shutdown_connections 2023-12-11T19:32:00.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.330+0000 7f48157ff700 1 -- 172.21.15.66:0/2841628146 wait complete. 2023-12-11T19:32:00.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.331+0000 7f48157ff700 1 Processor -- start 2023-12-11T19:32:00.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.331+0000 7f48157ff700 1 -- start start 2023-12-11T19:32:00.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f48157ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 0x7f4810197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f48157ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810102690 0x7f4810197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f48157ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4810103030 0x7f481019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f48157ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481019ef60 con 0x7f48101014a0 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f48157ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481019f0a0 con 0x7f4810103030 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f48157ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481019f230 con 0x7f4810102690 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 0x7f4810197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 0x7f4810197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34038/0 (socket says 172.21.15.66:34038) 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 -- 172.21.15.66:0/1084793586 learned_addr learned my addr 172.21.15.66:0/1084793586 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480f7fe700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4810103030 0x7f481019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480e7fc700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810102690 0x7f4810197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 -- 172.21.15.66:0/1084793586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810102690 msgr2=0x7f4810197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810102690 0x7f4810197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 -- 172.21.15.66:0/1084793586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4810103030 msgr2=0x7f481019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4810103030 0x7f481019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480effd700 1 -- 172.21.15.66:0/1084793586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808009d00 con 0x7f48101014a0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.332+0000 7f480e7fc700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810102690 0x7f4810197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.333+0000 7f480effd700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 0x7f4810197430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f480000d980 tx=0x7f480000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.333+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004020 con 0x7f48101014a0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.333+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4800009730 con 0x7f48101014a0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.333+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800003a00 con 0x7f48101014a0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.333+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481019f510 con 0x7f48101014a0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.333+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481019fa20 con 0x7f48101014a0 2023-12-11T19:32:00.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.334+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810108920 con 0x7f48101014a0 2023-12-11T19:32:00.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.336+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f48000098a0 con 0x7f48101014a0 2023-12-11T19:32:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.337+0000 7f47f7fff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f47f80a1590 0x7f47f80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.338+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48000cca50 con 0x7f48101014a0 2023-12-11T19:32:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.338+0000 7f480e7fc700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f47f80a1590 0x7f47f80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.338+0000 7f480e7fc700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f47f80a1590 0x7f47f80a3a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4804005950 tx=0x7f48040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:00.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.341+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4800096000 con 0x7f48101014a0 2023-12-11T19:32:00.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.559+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f481010cbb0 con 0x7f48101014a0 2023-12-11T19:32:00.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.560+0000 7f47f7fff700 1 -- 172.21.15.66:0/1084793586 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22011 (secure 0 0 0) 0x7f4800099620 con 0x7f48101014a0 2023-12-11T19:32:00.561 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:00.561 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:31:53.249050+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:32:00.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f47f80a1590 msgr2=0x7f47f80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f47f80a1590 0x7f47f80a3a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4804005950 tx=0x7f48040058e0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 msgr2=0x7f4810197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 0x7f4810197430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f480000d980 tx=0x7f480000dc90 comp rx=0 tx=0).stop 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 shutdown_connections 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f47f80a1590 0x7f47f80a3a50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4810103030 0x7f481019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48101014a0 0x7f4810197430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 --2- 172.21.15.66:0/1084793586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4810102690 0x7f4810197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 >> 172.21.15.66:0/1084793586 conn(0x7f48100fca10 msgr2=0x7f4810105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 shutdown_connections 2023-12-11T19:32:00.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.563+0000 7f48157ff700 1 -- 172.21.15.66:0/1084793586 wait complete. 2023-12-11T19:32:00.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-11T19:32:00.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T19:32:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 -- 172.21.15.66:0/585401061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318101ee0 msgr2=0x7f73181042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 --2- 172.21.15.66:0/585401061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318101ee0 0x7f73181042d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7314009990 tx=0x7f7314009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 -- 172.21.15.66:0/585401061 shutdown_connections 2023-12-11T19:32:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 --2- 172.21.15.66:0/585401061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 0x7f73181095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 --2- 172.21.15.66:0/585401061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318101ee0 0x7f73181042d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 --2- 172.21.15.66:0/585401061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7318104810 0x7f7318106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 -- 172.21.15.66:0/585401061 >> 172.21.15.66:0/585401061 conn(0x7f73180fb8b0 msgr2=0x7f73180fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 -- 172.21.15.66:0/585401061 shutdown_connections 2023-12-11T19:32:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.790+0000 7f731e79d700 1 -- 172.21.15.66:0/585401061 wait complete. 2023-12-11T19:32:00.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.791+0000 7f731e79d700 1 Processor -- start 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 -- start start 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7318101ee0 0x7f73181995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 0x7f7318199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 0x7f73181a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73181a10f0 con 0x7f7318104810 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73181a1230 con 0x7f7318101ee0 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731e79d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73181a13c0 con 0x7f7318107140 2023-12-11T19:32:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731cf9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 0x7f7318199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731cf9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 0x7f7318199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34058/0 (socket says 172.21.15.66:34058) 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731cf9a700 1 -- 172.21.15.66:0/4291876557 learned_addr learned my addr 172.21.15.66:0/4291876557 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.792+0000 7f731d79b700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7318101ee0 0x7f73181995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731df9c700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 0x7f73181a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731cf9a700 1 -- 172.21.15.66:0/4291876557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 msgr2=0x7f73181a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731cf9a700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 0x7f73181a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731cf9a700 1 -- 172.21.15.66:0/4291876557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7318101ee0 msgr2=0x7f73181995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731cf9a700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7318101ee0 0x7f73181995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731cf9a700 1 -- 172.21.15.66:0/4291876557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140095f0 con 0x7f7318104810 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731df9c700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 0x7f73181a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731cf9a700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 0x7f7318199b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f730800d980 tx=0x7f730800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308004220 con 0x7f7318104810 2023-12-11T19:32:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7308007640 con 0x7f7318104810 2023-12-11T19:32:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73181a16a0 con 0x7f7318104810 2023-12-11T19:32:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080055e0 con 0x7f7318104810 2023-12-11T19:32:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.793+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73181a1bb0 con 0x7f7318104810 2023-12-11T19:32:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.794+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7318108d40 con 0x7f7318104810 2023-12-11T19:32:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.796+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7308004750 con 0x7f7318104810 2023-12-11T19:32:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.798+0000 7f730e7fc700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f73040a1510 0x7f73040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.798+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73080ccc50 con 0x7f7318104810 2023-12-11T19:32:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.798+0000 7f731d79b700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f73040a1510 0x7f73040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.799+0000 7f731d79b700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f73040a1510 0x7f73040a39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7314003780 tx=0x7f7314005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:00.802+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7308096200 con 0x7f7318104810 2023-12-11T19:32:00.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:00 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:active 2023-12-11T19:32:00.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:00 smithi066 ceph-mon[97920]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-12-11T19:32:00.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:00 smithi066 ceph-mon[97920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:32:00.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:00 smithi066 ceph-mon[97920]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:32:00.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:00 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2523103056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:00.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:active 2023-12-11T19:32:00.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-mon[104044]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-12-11T19:32:00.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-mon[104044]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:32:00.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-mon[104044]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:32:00.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2523103056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:01.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.016+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7318061a10 con 0x7f73040a1510 2023-12-11T19:32:01.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:00 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:active 2023-12-11T19:32:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:00 smithi167 ceph-mon[102750]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-12-11T19:32:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:00 smithi167 ceph-mon[102750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:32:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:00 smithi167 ceph-mon[102750]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:32:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:00 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2523103056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:01.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.030+0000 7f730e7fc700 1 -- 172.21.15.66:0/4291876557 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334900 (secure 0 0 0) 0x7f7318061a10 con 0x7f73040a1510 2023-12-11T19:32:01.035 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f73040a1510 msgr2=0x7f73040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f73040a1510 0x7f73040a39d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7314003780 tx=0x7f7314005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 msgr2=0x7f7318199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 0x7f7318199b00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f730800d980 tx=0x7f730800dc90 comp rx=0 tx=0).stop 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 shutdown_connections 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f73040a1510 0x7f73040a39d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7318101ee0 0x7f73181995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7318107140 0x7f73181a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 --2- 172.21.15.66:0/4291876557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7318104810 0x7f7318199b00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 >> 172.21.15.66:0/4291876557 conn(0x7f73180fb8b0 msgr2=0x7f7318101690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.034+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 shutdown_connections 2023-12-11T19:32:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.035+0000 7f731e79d700 1 -- 172.21.15.66:0/4291876557 wait complete. 2023-12-11T19:32:01.039 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:32:01.055 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2023-12-11T19:32:00.286697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634091,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":155,"num_read_kb":165,"num_write":349,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":273,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335468,"kb_used_data":11268,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606356,"statfs":{"total":1151940427776,"available":1151596908544,"internally_reserved":0,"allocated":11538432,"data_stored":7122971,"data_compressed":110441,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43723,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":29,"num_read_kb":56,"num_write":168,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.037526"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977324+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:31:53.977324+0000","last_peered":"2023-12-11T19:31:53.977324+0000","last_clean":"2023-12-11T19:31:53.977324+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:31:53.977324+0000","last_undegraded":"2023-12-11T19:31:53.977324+0000","last_fullsized":"2023-12-11T19:31:53.977324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986885+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:31:53.986885+0000","last_peered":"2023-12-11T19:31:53.986885+0000","last_clean":"2023-12-11T19:31:53.986885+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:31:53.986885+0000","last_undegraded":"2023-12-11T19:31:53.986885+0000","last_fullsized":"2023-12-11T19:31:53.986885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986820+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:31:53.986820+0000","last_peered":"2023-12-11T19:31:53.986820+0000","last_clean":"2023-12-11T19:31:53.986820+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:31:53.986820+0000","last_undegraded":"2023-12-11T19:31:53.986820+0000","last_fullsized":"2023-12-11T19:31:53.986820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261637+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:31:53.261637+0000","last_peered":"2023-12-11T19:31:53.261637+0000","last_clean":"2023-12-11T19:31:53.261637+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:31:53.261637+0000","last_undegraded":"2023-12-11T19:31:53.261637+0000","last_fullsized":"2023-12-11T19:31:53.261637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977755+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:31:53.977755+0000","last_peered":"2023-12-11T19:31:53.977755+0000","last_clean":"2023-12-11T19:31:53.977755+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:31:53.977755+0000","last_undegraded":"2023-12-11T19:31:53.977755+0000","last_fullsized":"2023-12-11T19:31:53.977755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.730372+0000","last_change":"2023-12-11T19:31:52.265439+0000","last_active":"2023-12-11T19:31:53.730372+0000","last_peered":"2023-12-11T19:31:53.730372+0000","last_clean":"2023-12-11T19:31:53.730372+0000","last_became_active":"2023-12-11T19:31:52.265107+0000","last_became_peered":"2023-12-11T19:31:52.265107+0000","last_unstale":"2023-12-11T19:31:53.730372+0000","last_undegraded":"2023-12-11T19:31:53.730372+0000","last_fullsized":"2023-12-11T19:31:53.730372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:46:18.702094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260326+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:31:53.260326+0000","last_peered":"2023-12-11T19:31:53.260326+0000","last_clean":"2023-12-11T19:31:53.260326+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:31:53.260326+0000","last_undegraded":"2023-12-11T19:31:53.260326+0000","last_fullsized":"2023-12-11T19:31:53.260326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.263712+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:31:53.263712+0000","last_peered":"2023-12-11T19:31:53.263712+0000","last_clean":"2023-12-11T19:31:53.263712+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:31:53.263712+0000","last_undegraded":"2023-12-11T19:31:53.263712+0000","last_fullsized":"2023-12-11T19:31:53.263712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976536+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:31:53.976536+0000","last_peered":"2023-12-11T19:31:53.976536+0000","last_clean":"2023-12-11T19:31:53.976536+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:31:53.976536+0000","last_undegraded":"2023-12-11T19:31:53.976536+0000","last_fullsized":"2023-12-11T19:31:53.976536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.730395+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:31:53.730395+0000","last_peered":"2023-12-11T19:31:53.730395+0000","last_clean":"2023-12-11T19:31:53.730395+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:31:53.730395+0000","last_undegraded":"2023-12-11T19:31:53.730395+0000","last_fullsized":"2023-12-11T19:31:53.730395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260658+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:31:53.260658+0000","last_peered":"2023-12-11T19:31:53.260658+0000","last_clean":"2023-12-11T19:31:53.260658+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:31:53.260658+0000","last_undegraded":"2023-12-11T19:31:53.260658+0000","last_fullsized":"2023-12-11T19:31:53.260658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261525+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:31:53.261525+0000","last_peered":"2023-12-11T19:31:53.261525+0000","last_clean":"2023-12-11T19:31:53.261525+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:31:53.261525+0000","last_undegraded":"2023-12-11T19:31:53.261525+0000","last_fullsized":"2023-12-11T19:31:53.261525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976875+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:31:53.976875+0000","last_peered":"2023-12-11T19:31:53.976875+0000","last_clean":"2023-12-11T19:31:53.976875+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:31:53.976875+0000","last_undegraded":"2023-12-11T19:31:53.976875+0000","last_fullsized":"2023-12-11T19:31:53.976875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986435+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:31:53.986435+0000","last_peered":"2023-12-11T19:31:53.986435+0000","last_clean":"2023-12-11T19:31:53.986435+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:31:53.986435+0000","last_undegraded":"2023-12-11T19:31:53.986435+0000","last_fullsized":"2023-12-11T19:31:53.986435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986411+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:31:53.986411+0000","last_peered":"2023-12-11T19:31:53.986411+0000","last_clean":"2023-12-11T19:31:53.986411+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:31:53.986411+0000","last_undegraded":"2023-12-11T19:31:53.986411+0000","last_fullsized":"2023-12-11T19:31:53.986411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976151+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:31:53.976151+0000","last_peered":"2023-12-11T19:31:53.976151+0000","last_clean":"2023-12-11T19:31:53.976151+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:31:53.976151+0000","last_undegraded":"2023-12-11T19:31:53.976151+0000","last_fullsized":"2023-12-11T19:31:53.976151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977465+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:31:53.977465+0000","last_peered":"2023-12-11T19:31:53.977465+0000","last_clean":"2023-12-11T19:31:53.977465+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:31:53.977465+0000","last_undegraded":"2023-12-11T19:31:53.977465+0000","last_fullsized":"2023-12-11T19:31:53.977465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.687994+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:31:53.687994+0000","last_peered":"2023-12-11T19:31:53.687994+0000","last_clean":"2023-12-11T19:31:53.687994+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:31:53.687994+0000","last_undegraded":"2023-12-11T19:31:53.687994+0000","last_fullsized":"2023-12-11T19:31:53.687994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977856+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:31:53.977856+0000","last_peered":"2023-12-11T19:31:53.977856+0000","last_clean":"2023-12-11T19:31:53.977856+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:31:53.977856+0000","last_undegraded":"2023-12-11T19:31:53.977856+0000","last_fullsized":"2023-12-11T19:31:53.977856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.550961+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:31:58.550961+0000","last_peered":"2023-12-11T19:31:58.550961+0000","last_clean":"2023-12-11T19:31:58.550961+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:31:58.550961+0000","last_undegraded":"2023-12-11T19:31:58.550961+0000","last_fullsized":"2023-12-11T19:31:58.550961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264279+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:31:53.264279+0000","last_peered":"2023-12-11T19:31:53.264279+0000","last_clean":"2023-12-11T19:31:53.264279+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:31:53.264279+0000","last_undegraded":"2023-12-11T19:31:53.264279+0000","last_fullsized":"2023-12-11T19:31:53.264279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264340+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:31:53.264340+0000","last_peered":"2023-12-11T19:31:53.264340+0000","last_clean":"2023-12-11T19:31:53.264340+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:31:53.264340+0000","last_undegraded":"2023-12-11T19:31:53.264340+0000","last_fullsized":"2023-12-11T19:31:53.264340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.987033+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:31:53.987033+0000","last_peered":"2023-12-11T19:31:53.987033+0000","last_clean":"2023-12-11T19:31:53.987033+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:31:53.987033+0000","last_undegraded":"2023-12-11T19:31:53.987033+0000","last_fullsized":"2023-12-11T19:31:53.987033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.494182+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:31:52.494182+0000","last_peered":"2023-12-11T19:31:52.494182+0000","last_clean":"2023-12-11T19:31:52.494182+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:31:52.494182+0000","last_undegraded":"2023-12-11T19:31:52.494182+0000","last_fullsized":"2023-12-11T19:31:52.494182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976128+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:31:53.976128+0000","last_peered":"2023-12-11T19:31:53.976128+0000","last_clean":"2023-12-11T19:31:53.976128+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:31:53.976128+0000","last_undegraded":"2023-12-11T19:31:53.976128+0000","last_fullsized":"2023-12-11T19:31:53.976128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.493153+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:31:52.493153+0000","last_peered":"2023-12-11T19:31:52.493153+0000","last_clean":"2023-12-11T19:31:52.493153+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:31:52.493153+0000","last_undegraded":"2023-12-11T19:31:52.493153+0000","last_fullsized":"2023-12-11T19:31:52.493153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978475+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:31:53.978475+0000","last_peered":"2023-12-11T19:31:53.978475+0000","last_clean":"2023-12-11T19:31:53.978475+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:31:53.978475+0000","last_undegraded":"2023-12-11T19:31:53.978475+0000","last_fullsized":"2023-12-11T19:31:53.978475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.491919+0000","last_change":"2023-12-11T19:31:51.256538+0000","last_active":"2023-12-11T19:31:58.491919+0000","last_peered":"2023-12-11T19:31:58.491919+0000","last_clean":"2023-12-11T19:31:58.491919+0000","last_became_active":"2023-12-11T19:31:51.256193+0000","last_became_peered":"2023-12-11T19:31:51.256193+0000","last_unstale":"2023-12-11T19:31:58.491919+0000","last_undegraded":"2023-12-11T19:31:58.491919+0000","last_fullsized":"2023-12-11T19:31:58.491919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:56:40.078867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977478+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:31:53.977478+0000","last_peered":"2023-12-11T19:31:53.977478+0000","last_clean":"2023-12-11T19:31:53.977478+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:31:53.977478+0000","last_undegraded":"2023-12-11T19:31:53.977478+0000","last_fullsized":"2023-12-11T19:31:53.977478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.527209+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:31:57.527209+0000","last_peered":"2023-12-11T19:31:57.527209+0000","last_clean":"2023-12-11T19:31:57.527209+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:31:57.527209+0000","last_undegraded":"2023-12-11T19:31:57.527209+0000","last_fullsized":"2023-12-11T19:31:57.527209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978307+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:31:53.978307+0000","last_peered":"2023-12-11T19:31:53.978307+0000","last_clean":"2023-12-11T19:31:53.978307+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:31:53.978307+0000","last_undegraded":"2023-12-11T19:31:53.978307+0000","last_fullsized":"2023-12-11T19:31:53.978307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.251077+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:31:52.251077+0000","last_peered":"2023-12-11T19:31:52.251077+0000","last_clean":"2023-12-11T19:31:52.251077+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:31:52.251077+0000","last_undegraded":"2023-12-11T19:31:52.251077+0000","last_fullsized":"2023-12-11T19:31:52.251077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.263771+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:31:53.263771+0000","last_peered":"2023-12-11T19:31:53.263771+0000","last_clean":"2023-12-11T19:31:53.263771+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:31:53.263771+0000","last_undegraded":"2023-12-11T19:31:53.263771+0000","last_fullsized":"2023-12-11T19:31:53.263771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.525997+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:31:57.525997+0000","last_peered":"2023-12-11T19:31:57.525997+0000","last_clean":"2023-12-11T19:31:57.525997+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:31:57.525997+0000","last_undegraded":"2023-12-11T19:31:57.525997+0000","last_fullsized":"2023-12-11T19:31:57.525997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.687932+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:31:53.687932+0000","last_peered":"2023-12-11T19:31:53.687932+0000","last_clean":"2023-12-11T19:31:53.687932+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:31:53.687932+0000","last_undegraded":"2023-12-11T19:31:53.687932+0000","last_fullsized":"2023-12-11T19:31:53.687932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:59.493610+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:31:59.493610+0000","last_peered":"2023-12-11T19:31:59.493610+0000","last_clean":"2023-12-11T19:31:59.493610+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:31:59.493610+0000","last_undegraded":"2023-12-11T19:31:59.493610+0000","last_fullsized":"2023-12-11T19:31:59.493610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976436+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:31:53.976436+0000","last_peered":"2023-12-11T19:31:53.976436+0000","last_clean":"2023-12-11T19:31:53.976436+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:31:53.976436+0000","last_undegraded":"2023-12-11T19:31:53.976436+0000","last_fullsized":"2023-12-11T19:31:53.976436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.548741+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:31:58.548741+0000","last_peered":"2023-12-11T19:31:58.548741+0000","last_clean":"2023-12-11T19:31:58.548741+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:31:58.548741+0000","last_undegraded":"2023-12-11T19:31:58.548741+0000","last_fullsized":"2023-12-11T19:31:58.548741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.987012+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:31:53.987012+0000","last_peered":"2023-12-11T19:31:53.987012+0000","last_clean":"2023-12-11T19:31:53.987012+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:31:53.987012+0000","last_undegraded":"2023-12-11T19:31:53.987012+0000","last_fullsized":"2023-12-11T19:31:53.987012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:56.487772+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:31:56.487772+0000","last_peered":"2023-12-11T19:31:56.487772+0000","last_clean":"2023-12-11T19:31:56.487772+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:31:56.487772+0000","last_undegraded":"2023-12-11T19:31:56.487772+0000","last_fullsized":"2023-12-11T19:31:56.487772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976437+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:31:53.976437+0000","last_peered":"2023-12-11T19:31:53.976437+0000","last_clean":"2023-12-11T19:31:53.976437+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:31:53.976437+0000","last_undegraded":"2023-12-11T19:31:53.976437+0000","last_fullsized":"2023-12-11T19:31:53.976437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'20","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:59.494033+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:31:59.494033+0000","last_peered":"2023-12-11T19:31:59.494033+0000","last_clean":"2023-12-11T19:31:59.494033+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:31:59.494033+0000","last_undegraded":"2023-12-11T19:31:59.494033+0000","last_fullsized":"2023-12-11T19:31:59.494033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37719,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":42,"num_write":22,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978475+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:31:53.978475+0000","last_peered":"2023-12-11T19:31:53.978475+0000","last_clean":"2023-12-11T19:31:53.978475+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:31:53.978475+0000","last_undegraded":"2023-12-11T19:31:53.978475+0000","last_fullsized":"2023-12-11T19:31:53.978475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:56.499021+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:31:56.499021+0000","last_peered":"2023-12-11T19:31:56.499021+0000","last_clean":"2023-12-11T19:31:56.499021+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:31:56.499021+0000","last_undegraded":"2023-12-11T19:31:56.499021+0000","last_fullsized":"2023-12-11T19:31:56.499021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976915+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:31:53.976915+0000","last_peered":"2023-12-11T19:31:53.976915+0000","last_clean":"2023-12-11T19:31:53.976915+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:31:53.976915+0000","last_undegraded":"2023-12-11T19:31:53.976915+0000","last_fullsized":"2023-12-11T19:31:53.976915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.528608+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:31:57.528608+0000","last_peered":"2023-12-11T19:31:57.528608+0000","last_clean":"2023-12-11T19:31:57.528608+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:31:57.528608+0000","last_undegraded":"2023-12-11T19:31:57.528608+0000","last_fullsized":"2023-12-11T19:31:57.528608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260530+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:31:53.260530+0000","last_peered":"2023-12-11T19:31:53.260530+0000","last_clean":"2023-12-11T19:31:53.260530+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:31:53.260530+0000","last_undegraded":"2023-12-11T19:31:53.260530+0000","last_fullsized":"2023-12-11T19:31:53.260530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.525809+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:31:57.525809+0000","last_peered":"2023-12-11T19:31:57.525809+0000","last_clean":"2023-12-11T19:31:57.525809+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:31:57.525809+0000","last_undegraded":"2023-12-11T19:31:57.525809+0000","last_fullsized":"2023-12-11T19:31:57.525809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261049+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:31:53.261049+0000","last_peered":"2023-12-11T19:31:53.261049+0000","last_clean":"2023-12-11T19:31:53.261049+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:31:53.261049+0000","last_undegraded":"2023-12-11T19:31:53.261049+0000","last_fullsized":"2023-12-11T19:31:53.261049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.509819+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:31:55.509819+0000","last_peered":"2023-12-11T19:31:55.509819+0000","last_clean":"2023-12-11T19:31:55.509819+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:31:55.509819+0000","last_undegraded":"2023-12-11T19:31:55.509819+0000","last_fullsized":"2023-12-11T19:31:55.509819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976649+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:31:53.976649+0000","last_peered":"2023-12-11T19:31:53.976649+0000","last_clean":"2023-12-11T19:31:53.976649+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:31:53.976649+0000","last_undegraded":"2023-12-11T19:31:53.976649+0000","last_fullsized":"2023-12-11T19:31:53.976649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.491740+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:31:52.491740+0000","last_peered":"2023-12-11T19:31:52.491740+0000","last_clean":"2023-12-11T19:31:52.491740+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:31:52.491740+0000","last_undegraded":"2023-12-11T19:31:52.491740+0000","last_fullsized":"2023-12-11T19:31:52.491740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978053+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:31:53.978053+0000","last_peered":"2023-12-11T19:31:53.978053+0000","last_clean":"2023-12-11T19:31:53.978053+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:31:53.978053+0000","last_undegraded":"2023-12-11T19:31:53.978053+0000","last_fullsized":"2023-12-11T19:31:53.978053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.853772+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:31:53.853772+0000","last_peered":"2023-12-11T19:31:53.853772+0000","last_clean":"2023-12-11T19:31:53.853772+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:31:53.853772+0000","last_undegraded":"2023-12-11T19:31:53.853772+0000","last_fullsized":"2023-12-11T19:31:53.853772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.136964+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:31:55.136964+0000","last_peered":"2023-12-11T19:31:55.136964+0000","last_clean":"2023-12-11T19:31:55.136964+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:31:55.136964+0000","last_undegraded":"2023-12-11T19:31:55.136964+0000","last_fullsized":"2023-12-11T19:31:55.136964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.550830+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:31:58.550830+0000","last_peered":"2023-12-11T19:31:58.550830+0000","last_clean":"2023-12-11T19:31:58.550830+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:31:58.550830+0000","last_undegraded":"2023-12-11T19:31:58.550830+0000","last_fullsized":"2023-12-11T19:31:58.550830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.730317+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:31:53.730317+0000","last_peered":"2023-12-11T19:31:53.730317+0000","last_clean":"2023-12-11T19:31:53.730317+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:31:53.730317+0000","last_undegraded":"2023-12-11T19:31:53.730317+0000","last_fullsized":"2023-12-11T19:31:53.730317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976885+0000","last_change":"2023-12-11T19:31:52.267130+0000","last_active":"2023-12-11T19:31:53.976885+0000","last_peered":"2023-12-11T19:31:53.976885+0000","last_clean":"2023-12-11T19:31:53.976885+0000","last_became_active":"2023-12-11T19:31:52.266800+0000","last_became_peered":"2023-12-11T19:31:52.266800+0000","last_unstale":"2023-12-11T19:31:53.976885+0000","last_undegraded":"2023-12-11T19:31:53.976885+0000","last_fullsized":"2023-12-11T19:31:53.976885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:49:59.286637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976361+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:31:53.976361+0000","last_peered":"2023-12-11T19:31:53.976361+0000","last_clean":"2023-12-11T19:31:53.976361+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:31:53.976361+0000","last_undegraded":"2023-12-11T19:31:53.976361+0000","last_fullsized":"2023-12-11T19:31:53.976361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264076+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:31:53.264076+0000","last_peered":"2023-12-11T19:31:53.264076+0000","last_clean":"2023-12-11T19:31:53.264076+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:31:53.264076+0000","last_undegraded":"2023-12-11T19:31:53.264076+0000","last_fullsized":"2023-12-11T19:31:53.264076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.527062+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:31:57.527062+0000","last_peered":"2023-12-11T19:31:57.527062+0000","last_clean":"2023-12-11T19:31:57.527062+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:31:57.527062+0000","last_undegraded":"2023-12-11T19:31:57.527062+0000","last_fullsized":"2023-12-11T19:31:57.527062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":312,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.264259+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:31:52.264259+0000","last_peered":"2023-12-11T19:31:52.264259+0000","last_clean":"2023-12-11T19:31:52.264259+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:31:52.264259+0000","last_undegraded":"2023-12-11T19:31:52.264259+0000","last_fullsized":"2023-12-11T19:31:52.264259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976383+0000","last_change":"2023-12-11T19:31:52.263939+0000","last_active":"2023-12-11T19:31:53.976383+0000","last_peered":"2023-12-11T19:31:53.976383+0000","last_clean":"2023-12-11T19:31:53.976383+0000","last_became_active":"2023-12-11T19:31:52.263605+0000","last_became_peered":"2023-12-11T19:31:52.263605+0000","last_unstale":"2023-12-11T19:31:53.976383+0000","last_undegraded":"2023-12-11T19:31:53.976383+0000","last_fullsized":"2023-12-11T19:31:53.976383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:37:22.967213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:56.497427+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:31:56.497427+0000","last_peered":"2023-12-11T19:31:56.497427+0000","last_clean":"2023-12-11T19:31:56.497427+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:31:56.497427+0000","last_undegraded":"2023-12-11T19:31:56.497427+0000","last_fullsized":"2023-12-11T19:31:56.497427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978529+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:31:53.978529+0000","last_peered":"2023-12-11T19:31:53.978529+0000","last_clean":"2023-12-11T19:31:53.978529+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:31:53.978529+0000","last_undegraded":"2023-12-11T19:31:53.978529+0000","last_fullsized":"2023-12-11T19:31:53.978529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977088+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:31:53.977088+0000","last_peered":"2023-12-11T19:31:53.977088+0000","last_clean":"2023-12-11T19:31:53.977088+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:31:53.977088+0000","last_undegraded":"2023-12-11T19:31:53.977088+0000","last_fullsized":"2023-12-11T19:31:53.977088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.528416+0000","last_change":"2023-12-11T19:31:51.262324+0000","last_active":"2023-12-11T19:31:57.528416+0000","last_peered":"2023-12-11T19:31:57.528416+0000","last_clean":"2023-12-11T19:31:57.528416+0000","last_became_active":"2023-12-11T19:31:51.261927+0000","last_became_peered":"2023-12-11T19:31:51.261927+0000","last_unstale":"2023-12-11T19:31:57.528416+0000","last_undegraded":"2023-12-11T19:31:57.528416+0000","last_fullsized":"2023-12-11T19:31:57.528416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:28:43.634285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264567+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:31:53.264567+0000","last_peered":"2023-12-11T19:31:53.264567+0000","last_clean":"2023-12-11T19:31:53.264567+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:31:53.264567+0000","last_undegraded":"2023-12-11T19:31:53.264567+0000","last_fullsized":"2023-12-11T19:31:53.264567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.510299+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:31:55.510299+0000","last_peered":"2023-12-11T19:31:55.510299+0000","last_clean":"2023-12-11T19:31:55.510299+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:31:55.510299+0000","last_undegraded":"2023-12-11T19:31:55.510299+0000","last_fullsized":"2023-12-11T19:31:55.510299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986880+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:31:53.986880+0000","last_peered":"2023-12-11T19:31:53.986880+0000","last_clean":"2023-12-11T19:31:53.986880+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:31:53.986880+0000","last_undegraded":"2023-12-11T19:31:53.986880+0000","last_fullsized":"2023-12-11T19:31:53.986880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.510873+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:31:55.510873+0000","last_peered":"2023-12-11T19:31:55.510873+0000","last_clean":"2023-12-11T19:31:55.510873+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:31:55.510873+0000","last_undegraded":"2023-12-11T19:31:55.510873+0000","last_fullsized":"2023-12-11T19:31:55.510873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977370+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:31:53.977370+0000","last_peered":"2023-12-11T19:31:53.977370+0000","last_clean":"2023-12-11T19:31:53.977370+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:31:53.977370+0000","last_undegraded":"2023-12-11T19:31:53.977370+0000","last_fullsized":"2023-12-11T19:31:53.977370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.549732+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:31:58.549732+0000","last_peered":"2023-12-11T19:31:58.549732+0000","last_clean":"2023-12-11T19:31:58.549732+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:31:58.549732+0000","last_undegraded":"2023-12-11T19:31:58.549732+0000","last_fullsized":"2023-12-11T19:31:58.549732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976829+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:31:53.976829+0000","last_peered":"2023-12-11T19:31:53.976829+0000","last_clean":"2023-12-11T19:31:53.976829+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:31:53.976829+0000","last_undegraded":"2023-12-11T19:31:53.976829+0000","last_fullsized":"2023-12-11T19:31:53.976829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978410+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:31:53.978410+0000","last_peered":"2023-12-11T19:31:53.978410+0000","last_clean":"2023-12-11T19:31:53.978410+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:31:53.978410+0000","last_undegraded":"2023-12-11T19:31:53.978410+0000","last_fullsized":"2023-12-11T19:31:53.978410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264465+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:31:53.264465+0000","last_peered":"2023-12-11T19:31:53.264465+0000","last_clean":"2023-12-11T19:31:53.264465+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:31:53.264465+0000","last_undegraded":"2023-12-11T19:31:53.264465+0000","last_fullsized":"2023-12-11T19:31:53.264465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.509161+0000","last_change":"2023-12-11T19:31:51.266502+0000","last_active":"2023-12-11T19:31:55.509161+0000","last_peered":"2023-12-11T19:31:55.509161+0000","last_clean":"2023-12-11T19:31:55.509161+0000","last_became_active":"2023-12-11T19:31:51.266133+0000","last_became_peered":"2023-12-11T19:31:51.266133+0000","last_unstale":"2023-12-11T19:31:55.509161+0000","last_undegraded":"2023-12-11T19:31:55.509161+0000","last_fullsized":"2023-12-11T19:31:55.509161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:08:32.610640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260625+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:31:53.260625+0000","last_peered":"2023-12-11T19:31:53.260625+0000","last_clean":"2023-12-11T19:31:53.260625+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:31:53.260625+0000","last_undegraded":"2023-12-11T19:31:53.260625+0000","last_fullsized":"2023-12-11T19:31:53.260625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.526575+0000","last_change":"2023-12-11T19:31:51.261935+0000","last_active":"2023-12-11T19:31:57.526575+0000","last_peered":"2023-12-11T19:31:57.526575+0000","last_clean":"2023-12-11T19:31:57.526575+0000","last_became_active":"2023-12-11T19:31:51.255888+0000","last_became_peered":"2023-12-11T19:31:51.255888+0000","last_unstale":"2023-12-11T19:31:57.526575+0000","last_undegraded":"2023-12-11T19:31:57.526575+0000","last_fullsized":"2023-12-11T19:31:57.526575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:15:57.458294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261091+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:31:53.261091+0000","last_peered":"2023-12-11T19:31:53.261091+0000","last_clean":"2023-12-11T19:31:53.261091+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:31:53.261091+0000","last_undegraded":"2023-12-11T19:31:53.261091+0000","last_fullsized":"2023-12-11T19:31:53.261091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.527584+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:31:57.527584+0000","last_peered":"2023-12-11T19:31:57.527584+0000","last_clean":"2023-12-11T19:31:57.527584+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:31:57.527584+0000","last_undegraded":"2023-12-11T19:31:57.527584+0000","last_fullsized":"2023-12-11T19:31:57.527584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260448+0000","last_change":"2023-12-11T19:31:52.277197+0000","last_active":"2023-12-11T19:31:53.260448+0000","last_peered":"2023-12-11T19:31:53.260448+0000","last_clean":"2023-12-11T19:31:53.260448+0000","last_became_active":"2023-12-11T19:31:52.274522+0000","last_became_peered":"2023-12-11T19:31:52.274522+0000","last_unstale":"2023-12-11T19:31:53.260448+0000","last_undegraded":"2023-12-11T19:31:53.260448+0000","last_fullsized":"2023-12-11T19:31:53.260448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:02:22.788148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.493875+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:31:52.493875+0000","last_peered":"2023-12-11T19:31:52.493875+0000","last_clean":"2023-12-11T19:31:52.493875+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:31:52.493875+0000","last_undegraded":"2023-12-11T19:31:52.493875+0000","last_fullsized":"2023-12-11T19:31:52.493875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261320+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:31:53.261320+0000","last_peered":"2023-12-11T19:31:53.261320+0000","last_clean":"2023-12-11T19:31:53.261320+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:31:53.261320+0000","last_undegraded":"2023-12-11T19:31:53.261320+0000","last_fullsized":"2023-12-11T19:31:53.261320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976853+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:31:53.976853+0000","last_peered":"2023-12-11T19:31:53.976853+0000","last_clean":"2023-12-11T19:31:53.976853+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:31:53.976853+0000","last_undegraded":"2023-12-11T19:31:53.976853+0000","last_fullsized":"2023-12-11T19:31:53.976853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976552+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:31:53.976552+0000","last_peered":"2023-12-11T19:31:53.976552+0000","last_clean":"2023-12-11T19:31:53.976552+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:31:53.976552+0000","last_undegraded":"2023-12-11T19:31:53.976552+0000","last_fullsized":"2023-12-11T19:31:53.976552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977598+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:31:53.977598+0000","last_peered":"2023-12-11T19:31:53.977598+0000","last_clean":"2023-12-11T19:31:53.977598+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:31:53.977598+0000","last_undegraded":"2023-12-11T19:31:53.977598+0000","last_fullsized":"2023-12-11T19:31:53.977598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986769+0000","last_change":"2023-12-11T19:31:52.272992+0000","last_active":"2023-12-11T19:31:53.986769+0000","last_peered":"2023-12-11T19:31:53.986769+0000","last_clean":"2023-12-11T19:31:53.986769+0000","last_became_active":"2023-12-11T19:31:52.272088+0000","last_became_peered":"2023-12-11T19:31:52.272088+0000","last_unstale":"2023-12-11T19:31:53.986769+0000","last_undegraded":"2023-12-11T19:31:53.986769+0000","last_fullsized":"2023-12-11T19:31:53.986769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:37:03.906666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977003+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:31:53.977003+0000","last_peered":"2023-12-11T19:31:53.977003+0000","last_clean":"2023-12-11T19:31:53.977003+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:31:53.977003+0000","last_undegraded":"2023-12-11T19:31:53.977003+0000","last_fullsized":"2023-12-11T19:31:53.977003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977434+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:31:53.977434+0000","last_peered":"2023-12-11T19:31:53.977434+0000","last_clean":"2023-12-11T19:31:53.977434+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:31:53.977434+0000","last_undegraded":"2023-12-11T19:31:53.977434+0000","last_fullsized":"2023-12-11T19:31:53.977434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978193+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:31:53.978193+0000","last_peered":"2023-12-11T19:31:53.978193+0000","last_clean":"2023-12-11T19:31:53.978193+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:31:53.978193+0000","last_undegraded":"2023-12-11T19:31:53.978193+0000","last_fullsized":"2023-12-11T19:31:53.978193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264460+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:31:53.264460+0000","last_peered":"2023-12-11T19:31:53.264460+0000","last_clean":"2023-12-11T19:31:53.264460+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:31:53.264460+0000","last_undegraded":"2023-12-11T19:31:53.264460+0000","last_fullsized":"2023-12-11T19:31:53.264460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976839+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:31:53.976839+0000","last_peered":"2023-12-11T19:31:53.976839+0000","last_clean":"2023-12-11T19:31:53.976839+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:31:53.976839+0000","last_undegraded":"2023-12-11T19:31:53.976839+0000","last_fullsized":"2023-12-11T19:31:53.976839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.263840+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:31:53.263840+0000","last_peered":"2023-12-11T19:31:53.263840+0000","last_clean":"2023-12-11T19:31:53.263840+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:31:53.263840+0000","last_undegraded":"2023-12-11T19:31:53.263840+0000","last_fullsized":"2023-12-11T19:31:53.263840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976297+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:31:53.976297+0000","last_peered":"2023-12-11T19:31:53.976297+0000","last_clean":"2023-12-11T19:31:53.976297+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:31:53.976297+0000","last_undegraded":"2023-12-11T19:31:53.976297+0000","last_fullsized":"2023-12-11T19:31:53.976297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976848+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:31:53.976848+0000","last_peered":"2023-12-11T19:31:53.976848+0000","last_clean":"2023-12-11T19:31:53.976848+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:31:53.976848+0000","last_undegraded":"2023-12-11T19:31:53.976848+0000","last_fullsized":"2023-12-11T19:31:53.976848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976552+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:31:53.976552+0000","last_peered":"2023-12-11T19:31:53.976552+0000","last_clean":"2023-12-11T19:31:53.976552+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:31:53.976552+0000","last_undegraded":"2023-12-11T19:31:53.976552+0000","last_fullsized":"2023-12-11T19:31:53.976552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43600,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":29,"num_read_kb":56,"num_write":167,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":75414,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":441954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1071707,"data_compressed":38035,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.71499999999999997}]},{"osd":2,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.496}]},{"osd":4,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.439},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.85999999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":442280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.504}]},{"osd":1,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.72299999999999998}]},{"osd":7,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.74399999999999999}]},{"osd":8,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.78500000000000003}]}]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":441998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.39400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.46500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.72299999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.5}]},{"osd":3,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.62}]},{"osd":4,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":0.754},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.73599999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.90400000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.92600000000000005,"15min":0.92900000000000005},"min":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76300000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.94499999999999995}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.88200000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1033873,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85099999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39000000000000001}]},{"osd":2,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98399999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.101,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.046,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63200000000000001,"15min":0.64500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.874,"15min":0.874},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.238,"15min":1.238},"last":0.76400000000000001}]},{"osd":5,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":0.78400000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97999999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.224,"5min":1.28,"15min":1.28},"last":0.45900000000000002}]},{"osd":6,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1299999999999999,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1160000000000001,"5min":1.359,"15min":1.359},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.63},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.77000000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.274,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52600000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.751},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.73099999999999998}]},{"osd":10,"last update":"Mon Dec 11 19:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.64200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.31,"15min":0.31},"max":{"1min":0.68400000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.752}]},{"osd":2,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92500000000000004,"5min":1.284,"15min":1.284},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95599999999999996,"5min":1.228,"15min":1.228},"last":0.77700000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.016,"15min":1.016},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.119,"5min":1.157,"15min":1.157},"last":0.89200000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82399999999999995,"15min":0.81200000000000006},"min":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88600000000000001,"15min":0.92200000000000004},"min":{"1min":0.65400000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0349999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81799999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.873,"15min":0.86599999999999999},"min":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.92600000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.70999999999999996,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.204,"5min":1.212,"15min":1.212},"last":1.02},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.878,"15min":0.88300000000000001},"min":{"1min":0.64000000000000001,"5min":0.621,"15min":0.621},"max":{"1min":1.135,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.97599999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.081}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":442711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.64200000000000002,"15min":0.56599999999999995},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0469999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.77700000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.74299999999999999,"15min":0.70199999999999996},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.1180000000000003,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0349999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Dec 11 19:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.129,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.52400000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0229999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.85199999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.60399999999999998,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Dec 11 19:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.502,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93799999999999994,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81599999999999995,"15min":0.88200000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92200000000000004,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.63,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.871},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.05,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93000000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.438,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80300000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.78700000000000003},"min":{"1min":0.69299999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.96599999999999997}]},{"osd":9,"last update":"Mon Dec 11 19:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.98899999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.93999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":443438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.80400000000000005,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.05,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.64600000000000002},"min":{"1min":0.48899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.99399999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.502}]},{"osd":2,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.68899999999999995,"15min":0.627},"min":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0960000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.67400000000000004,"15min":0.65300000000000002},"min":{"1min":0.51000000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Dec 11 19:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.46100000000000002},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77200000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":1.004,"15min":1.004},"last":0.46100000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56299999999999994}]},{"osd":5,"last update":"Mon Dec 11 19:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.65600000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.036,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.224,"5min":1.819,"15min":1.819},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Dec 11 19:31:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.64000000000000001}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.71299999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.746},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.89600000000000002},"min":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.753}]},{"osd":9,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.77800000000000002}]},{"osd":10,"last update":"Mon Dec 11 19:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":444956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80500000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82099999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":1.115,"15min":1.115},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.877}]},{"osd":3,"last update":"Mon Dec 11 19:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85199999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66300000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.58999999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.745}]},{"osd":5,"last update":"Mon Dec 11 19:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.41299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82699999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.79700000000000004}]},{"osd":9,"last update":"Mon Dec 11 19:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93400000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.95399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":441717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.434,"15min":0.39400000000000002},"min":{"1min":0.40200000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.42499999999999999},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.755,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.626,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.504},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.52500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.68600000000000005,"15min":0.624},"min":{"1min":0.58599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.751},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68400000000000005,"15min":0.63400000000000001},"min":{"1min":0.53500000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.054,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.75600000000000001,"15min":0.69999999999999996},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.105,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.70999999999999996}]},{"osd":6,"last update":"Mon Dec 11 19:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.83399999999999996,"15min":0.81699999999999995},"min":{"1min":0.60799999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.67900000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.81299999999999994}]},{"osd":10,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93999999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95899999999999996,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.55100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97499999999999998,"5min":1.075,"15min":1.075},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Dec 11 19:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.498},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80300000000000005,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.80100000000000005,"15min":0.78600000000000003},"min":{"1min":0.61499999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.75600000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67100000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1034949,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.622},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.51500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.177,"15min":1.177},"last":0.66300000000000003}]},{"osd":1,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.48699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98999999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79900000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.47699999999999998,"15min":0.45800000000000002},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.75},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.50700000000000001},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.55500000000000005}]},{"osd":3,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.504},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94999999999999996,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.749,"15min":0.748},"min":{"1min":0.46000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95599999999999996,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2110000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76300000000000001,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.746},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.248,"5min":1.597,"15min":1.597},"last":0.59999999999999998}]},{"osd":7,"last update":"Mon Dec 11 19:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.749,"15min":0.73999999999999999},"min":{"1min":0.56100000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85899999999999999}]},{"osd":8,"last update":"Mon Dec 11 19:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.83099999999999996,"15min":0.82899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.129,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.75600000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":21337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":39844,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:32:01.064 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 -- 172.21.15.66:0/3094001792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 msgr2=0x7f1abc1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/3094001792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc1018c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00099c0 tx=0x7f1ab0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 -- 172.21.15.66:0/3094001792 shutdown_connections 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/3094001792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1abc103030 0x7f1abc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/3094001792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc1018c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/3094001792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1abc102690 0x7f1abc102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 -- 172.21.15.66:0/3094001792 >> 172.21.15.66:0/3094001792 conn(0x7f1abc0fca10 msgr2=0x7f1abc0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:01.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 -- 172.21.15.66:0/3094001792 shutdown_connections 2023-12-11T19:32:01.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.302+0000 7f1ac2bff700 1 -- 172.21.15.66:0/3094001792 wait complete. 2023-12-11T19:32:01.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 Processor -- start 2023-12-11T19:32:01.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 -- start start 2023-12-11T19:32:01.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:01.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1abc102690 0x7f1abc0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:01.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1abc103030 0x7f1abc073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:01.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19b230 con 0x7f1abc1014a0 2023-12-11T19:32:01.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc074440 con 0x7f1abc102690 2023-12-11T19:32:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.303+0000 7f1ac2bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc074600 con 0x7f1abc103030 2023-12-11T19:32:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34088/0 (socket says 172.21.15.66:34088) 2023-12-11T19:32:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 -- 172.21.15.66:0/821494636 learned_addr learned my addr 172.21.15.66:0/821494636 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1abbfff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1abc102690 0x7f1abc0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:01.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac119c700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1abc103030 0x7f1abc073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 -- 172.21.15.66:0/821494636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1abc103030 msgr2=0x7f1abc073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1abc103030 0x7f1abc073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 -- 172.21.15.66:0/821494636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1abc102690 msgr2=0x7f1abc0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1abc102690 0x7f1abc0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 -- 172.21.15.66:0/821494636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab00095f0 con 0x7f1abc1014a0 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.304+0000 7f1ac099b700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc075330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009fd0 tx=0x7f1ab000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.305+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab001d070 con 0x7f1abc1014a0 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.305+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc074820 con 0x7f1abc1014a0 2023-12-11T19:32:01.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.305+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ab000bd10 con 0x7f1abc1014a0 2023-12-11T19:32:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.305+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0004cb0 con 0x7f1abc1014a0 2023-12-11T19:32:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.305+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc1a5e80 con 0x7f1abc1014a0 2023-12-11T19:32:01.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.306+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc1089e0 con 0x7f1abc1014a0 2023-12-11T19:32:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.308+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1ab0005050 con 0x7f1abc1014a0 2023-12-11T19:32:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.309+0000 7f1ab9ffb700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1aac0a1510 0x7f1aac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.309+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ab00cdcb0 con 0x7f1abc1014a0 2023-12-11T19:32:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.309+0000 7f1abbfff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1aac0a1510 0x7f1aac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:01.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.310+0000 7f1abbfff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1aac0a1510 0x7f1aac0a39d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005950 tx=0x7f1aa80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:01.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.313+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1ab00972f0 con 0x7f1abc1014a0 2023-12-11T19:32:01.359 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b[140194]: 2023-12-11T19:32:00.942+0000 7f664657e700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-12-11T19:32:01.360 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:32:00 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b[140194]: 2023-12-11T19:32:00.942+0000 7f664657e700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-12-11T19:32:01.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.529+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1abc02d380 con 0x7f1aac0a1510 2023-12-11T19:32:01.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.540+0000 7f1ab9ffb700 1 -- 172.21.15.66:0/821494636 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334900 (secure 0 0 0) 0x7f1abc02d380 con 0x7f1aac0a1510 2023-12-11T19:32:01.545 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1aac0a1510 msgr2=0x7f1aac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1aac0a1510 0x7f1aac0a39d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005950 tx=0x7f1aa80058e0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 msgr2=0x7f1abc075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc075330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009fd0 tx=0x7f1ab000fab0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 shutdown_connections 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1aac0a1510 0x7f1aac0a39d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1abc103030 0x7f1abc073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1abc1014a0 0x7f1abc075330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 --2- 172.21.15.66:0/821494636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1abc102690 0x7f1abc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.544+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 >> 172.21.15.66:0/821494636 conn(0x7f1abc0fca10 msgr2=0x7f1abc105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.545+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 shutdown_connections 2023-12-11T19:32:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:01.545+0000 7f1ac2bff700 1 -- 172.21.15.66:0/821494636 wait complete. 2023-12-11T19:32:01.549 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:32:01.566 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2023-12-11T19:32:00.286697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634091,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":155,"num_read_kb":165,"num_write":349,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":273,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335468,"kb_used_data":11268,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606356,"statfs":{"total":1151940427776,"available":1151596908544,"internally_reserved":0,"allocated":11538432,"data_stored":7122971,"data_compressed":110441,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43723,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":29,"num_read_kb":56,"num_write":168,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.037526"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977324+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:31:53.977324+0000","last_peered":"2023-12-11T19:31:53.977324+0000","last_clean":"2023-12-11T19:31:53.977324+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:31:53.977324+0000","last_undegraded":"2023-12-11T19:31:53.977324+0000","last_fullsized":"2023-12-11T19:31:53.977324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986885+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:31:53.986885+0000","last_peered":"2023-12-11T19:31:53.986885+0000","last_clean":"2023-12-11T19:31:53.986885+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:31:53.986885+0000","last_undegraded":"2023-12-11T19:31:53.986885+0000","last_fullsized":"2023-12-11T19:31:53.986885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986820+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:31:53.986820+0000","last_peered":"2023-12-11T19:31:53.986820+0000","last_clean":"2023-12-11T19:31:53.986820+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:31:53.986820+0000","last_undegraded":"2023-12-11T19:31:53.986820+0000","last_fullsized":"2023-12-11T19:31:53.986820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261637+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:31:53.261637+0000","last_peered":"2023-12-11T19:31:53.261637+0000","last_clean":"2023-12-11T19:31:53.261637+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:31:53.261637+0000","last_undegraded":"2023-12-11T19:31:53.261637+0000","last_fullsized":"2023-12-11T19:31:53.261637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977755+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:31:53.977755+0000","last_peered":"2023-12-11T19:31:53.977755+0000","last_clean":"2023-12-11T19:31:53.977755+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:31:53.977755+0000","last_undegraded":"2023-12-11T19:31:53.977755+0000","last_fullsized":"2023-12-11T19:31:53.977755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.730372+0000","last_change":"2023-12-11T19:31:52.265439+0000","last_active":"2023-12-11T19:31:53.730372+0000","last_peered":"2023-12-11T19:31:53.730372+0000","last_clean":"2023-12-11T19:31:53.730372+0000","last_became_active":"2023-12-11T19:31:52.265107+0000","last_became_peered":"2023-12-11T19:31:52.265107+0000","last_unstale":"2023-12-11T19:31:53.730372+0000","last_undegraded":"2023-12-11T19:31:53.730372+0000","last_fullsized":"2023-12-11T19:31:53.730372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:46:18.702094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260326+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:31:53.260326+0000","last_peered":"2023-12-11T19:31:53.260326+0000","last_clean":"2023-12-11T19:31:53.260326+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:31:53.260326+0000","last_undegraded":"2023-12-11T19:31:53.260326+0000","last_fullsized":"2023-12-11T19:31:53.260326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.263712+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:31:53.263712+0000","last_peered":"2023-12-11T19:31:53.263712+0000","last_clean":"2023-12-11T19:31:53.263712+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:31:53.263712+0000","last_undegraded":"2023-12-11T19:31:53.263712+0000","last_fullsized":"2023-12-11T19:31:53.263712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976536+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:31:53.976536+0000","last_peered":"2023-12-11T19:31:53.976536+0000","last_clean":"2023-12-11T19:31:53.976536+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:31:53.976536+0000","last_undegraded":"2023-12-11T19:31:53.976536+0000","last_fullsized":"2023-12-11T19:31:53.976536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.730395+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:31:53.730395+0000","last_peered":"2023-12-11T19:31:53.730395+0000","last_clean":"2023-12-11T19:31:53.730395+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:31:53.730395+0000","last_undegraded":"2023-12-11T19:31:53.730395+0000","last_fullsized":"2023-12-11T19:31:53.730395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260658+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:31:53.260658+0000","last_peered":"2023-12-11T19:31:53.260658+0000","last_clean":"2023-12-11T19:31:53.260658+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:31:53.260658+0000","last_undegraded":"2023-12-11T19:31:53.260658+0000","last_fullsized":"2023-12-11T19:31:53.260658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261525+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:31:53.261525+0000","last_peered":"2023-12-11T19:31:53.261525+0000","last_clean":"2023-12-11T19:31:53.261525+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:31:53.261525+0000","last_undegraded":"2023-12-11T19:31:53.261525+0000","last_fullsized":"2023-12-11T19:31:53.261525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976875+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:31:53.976875+0000","last_peered":"2023-12-11T19:31:53.976875+0000","last_clean":"2023-12-11T19:31:53.976875+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:31:53.976875+0000","last_undegraded":"2023-12-11T19:31:53.976875+0000","last_fullsized":"2023-12-11T19:31:53.976875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986435+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:31:53.986435+0000","last_peered":"2023-12-11T19:31:53.986435+0000","last_clean":"2023-12-11T19:31:53.986435+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:31:53.986435+0000","last_undegraded":"2023-12-11T19:31:53.986435+0000","last_fullsized":"2023-12-11T19:31:53.986435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986411+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:31:53.986411+0000","last_peered":"2023-12-11T19:31:53.986411+0000","last_clean":"2023-12-11T19:31:53.986411+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:31:53.986411+0000","last_undegraded":"2023-12-11T19:31:53.986411+0000","last_fullsized":"2023-12-11T19:31:53.986411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976151+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:31:53.976151+0000","last_peered":"2023-12-11T19:31:53.976151+0000","last_clean":"2023-12-11T19:31:53.976151+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:31:53.976151+0000","last_undegraded":"2023-12-11T19:31:53.976151+0000","last_fullsized":"2023-12-11T19:31:53.976151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977465+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:31:53.977465+0000","last_peered":"2023-12-11T19:31:53.977465+0000","last_clean":"2023-12-11T19:31:53.977465+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:31:53.977465+0000","last_undegraded":"2023-12-11T19:31:53.977465+0000","last_fullsized":"2023-12-11T19:31:53.977465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.687994+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:31:53.687994+0000","last_peered":"2023-12-11T19:31:53.687994+0000","last_clean":"2023-12-11T19:31:53.687994+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:31:53.687994+0000","last_undegraded":"2023-12-11T19:31:53.687994+0000","last_fullsized":"2023-12-11T19:31:53.687994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977856+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:31:53.977856+0000","last_peered":"2023-12-11T19:31:53.977856+0000","last_clean":"2023-12-11T19:31:53.977856+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:31:53.977856+0000","last_undegraded":"2023-12-11T19:31:53.977856+0000","last_fullsized":"2023-12-11T19:31:53.977856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.550961+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:31:58.550961+0000","last_peered":"2023-12-11T19:31:58.550961+0000","last_clean":"2023-12-11T19:31:58.550961+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:31:58.550961+0000","last_undegraded":"2023-12-11T19:31:58.550961+0000","last_fullsized":"2023-12-11T19:31:58.550961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264279+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:31:53.264279+0000","last_peered":"2023-12-11T19:31:53.264279+0000","last_clean":"2023-12-11T19:31:53.264279+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:31:53.264279+0000","last_undegraded":"2023-12-11T19:31:53.264279+0000","last_fullsized":"2023-12-11T19:31:53.264279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264340+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:31:53.264340+0000","last_peered":"2023-12-11T19:31:53.264340+0000","last_clean":"2023-12-11T19:31:53.264340+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:31:53.264340+0000","last_undegraded":"2023-12-11T19:31:53.264340+0000","last_fullsized":"2023-12-11T19:31:53.264340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.987033+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:31:53.987033+0000","last_peered":"2023-12-11T19:31:53.987033+0000","last_clean":"2023-12-11T19:31:53.987033+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:31:53.987033+0000","last_undegraded":"2023-12-11T19:31:53.987033+0000","last_fullsized":"2023-12-11T19:31:53.987033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.494182+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:31:52.494182+0000","last_peered":"2023-12-11T19:31:52.494182+0000","last_clean":"2023-12-11T19:31:52.494182+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:31:52.494182+0000","last_undegraded":"2023-12-11T19:31:52.494182+0000","last_fullsized":"2023-12-11T19:31:52.494182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976128+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:31:53.976128+0000","last_peered":"2023-12-11T19:31:53.976128+0000","last_clean":"2023-12-11T19:31:53.976128+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:31:53.976128+0000","last_undegraded":"2023-12-11T19:31:53.976128+0000","last_fullsized":"2023-12-11T19:31:53.976128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.493153+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:31:52.493153+0000","last_peered":"2023-12-11T19:31:52.493153+0000","last_clean":"2023-12-11T19:31:52.493153+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:31:52.493153+0000","last_undegraded":"2023-12-11T19:31:52.493153+0000","last_fullsized":"2023-12-11T19:31:52.493153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978475+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:31:53.978475+0000","last_peered":"2023-12-11T19:31:53.978475+0000","last_clean":"2023-12-11T19:31:53.978475+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:31:53.978475+0000","last_undegraded":"2023-12-11T19:31:53.978475+0000","last_fullsized":"2023-12-11T19:31:53.978475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.491919+0000","last_change":"2023-12-11T19:31:51.256538+0000","last_active":"2023-12-11T19:31:58.491919+0000","last_peered":"2023-12-11T19:31:58.491919+0000","last_clean":"2023-12-11T19:31:58.491919+0000","last_became_active":"2023-12-11T19:31:51.256193+0000","last_became_peered":"2023-12-11T19:31:51.256193+0000","last_unstale":"2023-12-11T19:31:58.491919+0000","last_undegraded":"2023-12-11T19:31:58.491919+0000","last_fullsized":"2023-12-11T19:31:58.491919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:56:40.078867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977478+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:31:53.977478+0000","last_peered":"2023-12-11T19:31:53.977478+0000","last_clean":"2023-12-11T19:31:53.977478+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:31:53.977478+0000","last_undegraded":"2023-12-11T19:31:53.977478+0000","last_fullsized":"2023-12-11T19:31:53.977478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.527209+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:31:57.527209+0000","last_peered":"2023-12-11T19:31:57.527209+0000","last_clean":"2023-12-11T19:31:57.527209+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:31:57.527209+0000","last_undegraded":"2023-12-11T19:31:57.527209+0000","last_fullsized":"2023-12-11T19:31:57.527209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978307+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:31:53.978307+0000","last_peered":"2023-12-11T19:31:53.978307+0000","last_clean":"2023-12-11T19:31:53.978307+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:31:53.978307+0000","last_undegraded":"2023-12-11T19:31:53.978307+0000","last_fullsized":"2023-12-11T19:31:53.978307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.251077+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:31:52.251077+0000","last_peered":"2023-12-11T19:31:52.251077+0000","last_clean":"2023-12-11T19:31:52.251077+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:31:52.251077+0000","last_undegraded":"2023-12-11T19:31:52.251077+0000","last_fullsized":"2023-12-11T19:31:52.251077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.263771+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:31:53.263771+0000","last_peered":"2023-12-11T19:31:53.263771+0000","last_clean":"2023-12-11T19:31:53.263771+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:31:53.263771+0000","last_undegraded":"2023-12-11T19:31:53.263771+0000","last_fullsized":"2023-12-11T19:31:53.263771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.525997+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:31:57.525997+0000","last_peered":"2023-12-11T19:31:57.525997+0000","last_clean":"2023-12-11T19:31:57.525997+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:31:57.525997+0000","last_undegraded":"2023-12-11T19:31:57.525997+0000","last_fullsized":"2023-12-11T19:31:57.525997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.687932+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:31:53.687932+0000","last_peered":"2023-12-11T19:31:53.687932+0000","last_clean":"2023-12-11T19:31:53.687932+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:31:53.687932+0000","last_undegraded":"2023-12-11T19:31:53.687932+0000","last_fullsized":"2023-12-11T19:31:53.687932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:59.493610+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:31:59.493610+0000","last_peered":"2023-12-11T19:31:59.493610+0000","last_clean":"2023-12-11T19:31:59.493610+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:31:59.493610+0000","last_undegraded":"2023-12-11T19:31:59.493610+0000","last_fullsized":"2023-12-11T19:31:59.493610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976436+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:31:53.976436+0000","last_peered":"2023-12-11T19:31:53.976436+0000","last_clean":"2023-12-11T19:31:53.976436+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:31:53.976436+0000","last_undegraded":"2023-12-11T19:31:53.976436+0000","last_fullsized":"2023-12-11T19:31:53.976436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.548741+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:31:58.548741+0000","last_peered":"2023-12-11T19:31:58.548741+0000","last_clean":"2023-12-11T19:31:58.548741+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:31:58.548741+0000","last_undegraded":"2023-12-11T19:31:58.548741+0000","last_fullsized":"2023-12-11T19:31:58.548741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.987012+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:31:53.987012+0000","last_peered":"2023-12-11T19:31:53.987012+0000","last_clean":"2023-12-11T19:31:53.987012+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:31:53.987012+0000","last_undegraded":"2023-12-11T19:31:53.987012+0000","last_fullsized":"2023-12-11T19:31:53.987012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:56.487772+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:31:56.487772+0000","last_peered":"2023-12-11T19:31:56.487772+0000","last_clean":"2023-12-11T19:31:56.487772+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:31:56.487772+0000","last_undegraded":"2023-12-11T19:31:56.487772+0000","last_fullsized":"2023-12-11T19:31:56.487772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976437+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:31:53.976437+0000","last_peered":"2023-12-11T19:31:53.976437+0000","last_clean":"2023-12-11T19:31:53.976437+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:31:53.976437+0000","last_undegraded":"2023-12-11T19:31:53.976437+0000","last_fullsized":"2023-12-11T19:31:53.976437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'20","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:59.494033+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:31:59.494033+0000","last_peered":"2023-12-11T19:31:59.494033+0000","last_clean":"2023-12-11T19:31:59.494033+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:31:59.494033+0000","last_undegraded":"2023-12-11T19:31:59.494033+0000","last_fullsized":"2023-12-11T19:31:59.494033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37719,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":42,"num_write":22,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978475+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:31:53.978475+0000","last_peered":"2023-12-11T19:31:53.978475+0000","last_clean":"2023-12-11T19:31:53.978475+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:31:53.978475+0000","last_undegraded":"2023-12-11T19:31:53.978475+0000","last_fullsized":"2023-12-11T19:31:53.978475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'7","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:56.499021+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:31:56.499021+0000","last_peered":"2023-12-11T19:31:56.499021+0000","last_clean":"2023-12-11T19:31:56.499021+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:31:56.499021+0000","last_undegraded":"2023-12-11T19:31:56.499021+0000","last_fullsized":"2023-12-11T19:31:56.499021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976915+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:31:53.976915+0000","last_peered":"2023-12-11T19:31:53.976915+0000","last_clean":"2023-12-11T19:31:53.976915+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:31:53.976915+0000","last_undegraded":"2023-12-11T19:31:53.976915+0000","last_fullsized":"2023-12-11T19:31:53.976915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.528608+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:31:57.528608+0000","last_peered":"2023-12-11T19:31:57.528608+0000","last_clean":"2023-12-11T19:31:57.528608+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:31:57.528608+0000","last_undegraded":"2023-12-11T19:31:57.528608+0000","last_fullsized":"2023-12-11T19:31:57.528608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260530+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:31:53.260530+0000","last_peered":"2023-12-11T19:31:53.260530+0000","last_clean":"2023-12-11T19:31:53.260530+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:31:53.260530+0000","last_undegraded":"2023-12-11T19:31:53.260530+0000","last_fullsized":"2023-12-11T19:31:53.260530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.525809+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:31:57.525809+0000","last_peered":"2023-12-11T19:31:57.525809+0000","last_clean":"2023-12-11T19:31:57.525809+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:31:57.525809+0000","last_undegraded":"2023-12-11T19:31:57.525809+0000","last_fullsized":"2023-12-11T19:31:57.525809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261049+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:31:53.261049+0000","last_peered":"2023-12-11T19:31:53.261049+0000","last_clean":"2023-12-11T19:31:53.261049+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:31:53.261049+0000","last_undegraded":"2023-12-11T19:31:53.261049+0000","last_fullsized":"2023-12-11T19:31:53.261049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.509819+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:31:55.509819+0000","last_peered":"2023-12-11T19:31:55.509819+0000","last_clean":"2023-12-11T19:31:55.509819+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:31:55.509819+0000","last_undegraded":"2023-12-11T19:31:55.509819+0000","last_fullsized":"2023-12-11T19:31:55.509819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976649+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:31:53.976649+0000","last_peered":"2023-12-11T19:31:53.976649+0000","last_clean":"2023-12-11T19:31:53.976649+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:31:53.976649+0000","last_undegraded":"2023-12-11T19:31:53.976649+0000","last_fullsized":"2023-12-11T19:31:53.976649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.491740+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:31:52.491740+0000","last_peered":"2023-12-11T19:31:52.491740+0000","last_clean":"2023-12-11T19:31:52.491740+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:31:52.491740+0000","last_undegraded":"2023-12-11T19:31:52.491740+0000","last_fullsized":"2023-12-11T19:31:52.491740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978053+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:31:53.978053+0000","last_peered":"2023-12-11T19:31:53.978053+0000","last_clean":"2023-12-11T19:31:53.978053+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:31:53.978053+0000","last_undegraded":"2023-12-11T19:31:53.978053+0000","last_fullsized":"2023-12-11T19:31:53.978053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.853772+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:31:53.853772+0000","last_peered":"2023-12-11T19:31:53.853772+0000","last_clean":"2023-12-11T19:31:53.853772+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:31:53.853772+0000","last_undegraded":"2023-12-11T19:31:53.853772+0000","last_fullsized":"2023-12-11T19:31:53.853772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.136964+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:31:55.136964+0000","last_peered":"2023-12-11T19:31:55.136964+0000","last_clean":"2023-12-11T19:31:55.136964+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:31:55.136964+0000","last_undegraded":"2023-12-11T19:31:55.136964+0000","last_fullsized":"2023-12-11T19:31:55.136964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.550830+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:31:58.550830+0000","last_peered":"2023-12-11T19:31:58.550830+0000","last_clean":"2023-12-11T19:31:58.550830+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:31:58.550830+0000","last_undegraded":"2023-12-11T19:31:58.550830+0000","last_fullsized":"2023-12-11T19:31:58.550830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.730317+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:31:53.730317+0000","last_peered":"2023-12-11T19:31:53.730317+0000","last_clean":"2023-12-11T19:31:53.730317+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:31:53.730317+0000","last_undegraded":"2023-12-11T19:31:53.730317+0000","last_fullsized":"2023-12-11T19:31:53.730317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976885+0000","last_change":"2023-12-11T19:31:52.267130+0000","last_active":"2023-12-11T19:31:53.976885+0000","last_peered":"2023-12-11T19:31:53.976885+0000","last_clean":"2023-12-11T19:31:53.976885+0000","last_became_active":"2023-12-11T19:31:52.266800+0000","last_became_peered":"2023-12-11T19:31:52.266800+0000","last_unstale":"2023-12-11T19:31:53.976885+0000","last_undegraded":"2023-12-11T19:31:53.976885+0000","last_fullsized":"2023-12-11T19:31:53.976885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:49:59.286637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976361+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:31:53.976361+0000","last_peered":"2023-12-11T19:31:53.976361+0000","last_clean":"2023-12-11T19:31:53.976361+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:31:53.976361+0000","last_undegraded":"2023-12-11T19:31:53.976361+0000","last_fullsized":"2023-12-11T19:31:53.976361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264076+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:31:53.264076+0000","last_peered":"2023-12-11T19:31:53.264076+0000","last_clean":"2023-12-11T19:31:53.264076+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:31:53.264076+0000","last_undegraded":"2023-12-11T19:31:53.264076+0000","last_fullsized":"2023-12-11T19:31:53.264076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.527062+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:31:57.527062+0000","last_peered":"2023-12-11T19:31:57.527062+0000","last_clean":"2023-12-11T19:31:57.527062+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:31:57.527062+0000","last_undegraded":"2023-12-11T19:31:57.527062+0000","last_fullsized":"2023-12-11T19:31:57.527062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":312,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.264259+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:31:52.264259+0000","last_peered":"2023-12-11T19:31:52.264259+0000","last_clean":"2023-12-11T19:31:52.264259+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:31:52.264259+0000","last_undegraded":"2023-12-11T19:31:52.264259+0000","last_fullsized":"2023-12-11T19:31:52.264259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976383+0000","last_change":"2023-12-11T19:31:52.263939+0000","last_active":"2023-12-11T19:31:53.976383+0000","last_peered":"2023-12-11T19:31:53.976383+0000","last_clean":"2023-12-11T19:31:53.976383+0000","last_became_active":"2023-12-11T19:31:52.263605+0000","last_became_peered":"2023-12-11T19:31:52.263605+0000","last_unstale":"2023-12-11T19:31:53.976383+0000","last_undegraded":"2023-12-11T19:31:53.976383+0000","last_fullsized":"2023-12-11T19:31:53.976383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:37:22.967213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:56.497427+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:31:56.497427+0000","last_peered":"2023-12-11T19:31:56.497427+0000","last_clean":"2023-12-11T19:31:56.497427+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:31:56.497427+0000","last_undegraded":"2023-12-11T19:31:56.497427+0000","last_fullsized":"2023-12-11T19:31:56.497427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978529+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:31:53.978529+0000","last_peered":"2023-12-11T19:31:53.978529+0000","last_clean":"2023-12-11T19:31:53.978529+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:31:53.978529+0000","last_undegraded":"2023-12-11T19:31:53.978529+0000","last_fullsized":"2023-12-11T19:31:53.978529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977088+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:31:53.977088+0000","last_peered":"2023-12-11T19:31:53.977088+0000","last_clean":"2023-12-11T19:31:53.977088+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:31:53.977088+0000","last_undegraded":"2023-12-11T19:31:53.977088+0000","last_fullsized":"2023-12-11T19:31:53.977088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.528416+0000","last_change":"2023-12-11T19:31:51.262324+0000","last_active":"2023-12-11T19:31:57.528416+0000","last_peered":"2023-12-11T19:31:57.528416+0000","last_clean":"2023-12-11T19:31:57.528416+0000","last_became_active":"2023-12-11T19:31:51.261927+0000","last_became_peered":"2023-12-11T19:31:51.261927+0000","last_unstale":"2023-12-11T19:31:57.528416+0000","last_undegraded":"2023-12-11T19:31:57.528416+0000","last_fullsized":"2023-12-11T19:31:57.528416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:28:43.634285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264567+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:31:53.264567+0000","last_peered":"2023-12-11T19:31:53.264567+0000","last_clean":"2023-12-11T19:31:53.264567+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:31:53.264567+0000","last_undegraded":"2023-12-11T19:31:53.264567+0000","last_fullsized":"2023-12-11T19:31:53.264567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.510299+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:31:55.510299+0000","last_peered":"2023-12-11T19:31:55.510299+0000","last_clean":"2023-12-11T19:31:55.510299+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:31:55.510299+0000","last_undegraded":"2023-12-11T19:31:55.510299+0000","last_fullsized":"2023-12-11T19:31:55.510299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986880+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:31:53.986880+0000","last_peered":"2023-12-11T19:31:53.986880+0000","last_clean":"2023-12-11T19:31:53.986880+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:31:53.986880+0000","last_undegraded":"2023-12-11T19:31:53.986880+0000","last_fullsized":"2023-12-11T19:31:53.986880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.510873+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:31:55.510873+0000","last_peered":"2023-12-11T19:31:55.510873+0000","last_clean":"2023-12-11T19:31:55.510873+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:31:55.510873+0000","last_undegraded":"2023-12-11T19:31:55.510873+0000","last_fullsized":"2023-12-11T19:31:55.510873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977370+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:31:53.977370+0000","last_peered":"2023-12-11T19:31:53.977370+0000","last_clean":"2023-12-11T19:31:53.977370+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:31:53.977370+0000","last_undegraded":"2023-12-11T19:31:53.977370+0000","last_fullsized":"2023-12-11T19:31:53.977370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:58.549732+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:31:58.549732+0000","last_peered":"2023-12-11T19:31:58.549732+0000","last_clean":"2023-12-11T19:31:58.549732+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:31:58.549732+0000","last_undegraded":"2023-12-11T19:31:58.549732+0000","last_fullsized":"2023-12-11T19:31:58.549732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976829+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:31:53.976829+0000","last_peered":"2023-12-11T19:31:53.976829+0000","last_clean":"2023-12-11T19:31:53.976829+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:31:53.976829+0000","last_undegraded":"2023-12-11T19:31:53.976829+0000","last_fullsized":"2023-12-11T19:31:53.976829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978410+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:31:53.978410+0000","last_peered":"2023-12-11T19:31:53.978410+0000","last_clean":"2023-12-11T19:31:53.978410+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:31:53.978410+0000","last_undegraded":"2023-12-11T19:31:53.978410+0000","last_fullsized":"2023-12-11T19:31:53.978410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264465+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:31:53.264465+0000","last_peered":"2023-12-11T19:31:53.264465+0000","last_clean":"2023-12-11T19:31:53.264465+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:31:53.264465+0000","last_undegraded":"2023-12-11T19:31:53.264465+0000","last_fullsized":"2023-12-11T19:31:53.264465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:55.509161+0000","last_change":"2023-12-11T19:31:51.266502+0000","last_active":"2023-12-11T19:31:55.509161+0000","last_peered":"2023-12-11T19:31:55.509161+0000","last_clean":"2023-12-11T19:31:55.509161+0000","last_became_active":"2023-12-11T19:31:51.266133+0000","last_became_peered":"2023-12-11T19:31:51.266133+0000","last_unstale":"2023-12-11T19:31:55.509161+0000","last_undegraded":"2023-12-11T19:31:55.509161+0000","last_fullsized":"2023-12-11T19:31:55.509161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:08:32.610640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260625+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:31:53.260625+0000","last_peered":"2023-12-11T19:31:53.260625+0000","last_clean":"2023-12-11T19:31:53.260625+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:31:53.260625+0000","last_undegraded":"2023-12-11T19:31:53.260625+0000","last_fullsized":"2023-12-11T19:31:53.260625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.526575+0000","last_change":"2023-12-11T19:31:51.261935+0000","last_active":"2023-12-11T19:31:57.526575+0000","last_peered":"2023-12-11T19:31:57.526575+0000","last_clean":"2023-12-11T19:31:57.526575+0000","last_became_active":"2023-12-11T19:31:51.255888+0000","last_became_peered":"2023-12-11T19:31:51.255888+0000","last_unstale":"2023-12-11T19:31:57.526575+0000","last_undegraded":"2023-12-11T19:31:57.526575+0000","last_fullsized":"2023-12-11T19:31:57.526575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:15:57.458294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261091+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:31:53.261091+0000","last_peered":"2023-12-11T19:31:53.261091+0000","last_clean":"2023-12-11T19:31:53.261091+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:31:53.261091+0000","last_undegraded":"2023-12-11T19:31:53.261091+0000","last_fullsized":"2023-12-11T19:31:53.261091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:57.527584+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:31:57.527584+0000","last_peered":"2023-12-11T19:31:57.527584+0000","last_clean":"2023-12-11T19:31:57.527584+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:31:57.527584+0000","last_undegraded":"2023-12-11T19:31:57.527584+0000","last_fullsized":"2023-12-11T19:31:57.527584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.260448+0000","last_change":"2023-12-11T19:31:52.277197+0000","last_active":"2023-12-11T19:31:53.260448+0000","last_peered":"2023-12-11T19:31:53.260448+0000","last_clean":"2023-12-11T19:31:53.260448+0000","last_became_active":"2023-12-11T19:31:52.274522+0000","last_became_peered":"2023-12-11T19:31:52.274522+0000","last_unstale":"2023-12-11T19:31:53.260448+0000","last_undegraded":"2023-12-11T19:31:53.260448+0000","last_fullsized":"2023-12-11T19:31:53.260448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:02:22.788148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T19:31:52.493875+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:31:52.493875+0000","last_peered":"2023-12-11T19:31:52.493875+0000","last_clean":"2023-12-11T19:31:52.493875+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:31:52.493875+0000","last_undegraded":"2023-12-11T19:31:52.493875+0000","last_fullsized":"2023-12-11T19:31:52.493875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.261320+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:31:53.261320+0000","last_peered":"2023-12-11T19:31:53.261320+0000","last_clean":"2023-12-11T19:31:53.261320+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:31:53.261320+0000","last_undegraded":"2023-12-11T19:31:53.261320+0000","last_fullsized":"2023-12-11T19:31:53.261320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976853+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:31:53.976853+0000","last_peered":"2023-12-11T19:31:53.976853+0000","last_clean":"2023-12-11T19:31:53.976853+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:31:53.976853+0000","last_undegraded":"2023-12-11T19:31:53.976853+0000","last_fullsized":"2023-12-11T19:31:53.976853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976552+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:31:53.976552+0000","last_peered":"2023-12-11T19:31:53.976552+0000","last_clean":"2023-12-11T19:31:53.976552+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:31:53.976552+0000","last_undegraded":"2023-12-11T19:31:53.976552+0000","last_fullsized":"2023-12-11T19:31:53.976552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977598+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:31:53.977598+0000","last_peered":"2023-12-11T19:31:53.977598+0000","last_clean":"2023-12-11T19:31:53.977598+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:31:53.977598+0000","last_undegraded":"2023-12-11T19:31:53.977598+0000","last_fullsized":"2023-12-11T19:31:53.977598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.986769+0000","last_change":"2023-12-11T19:31:52.272992+0000","last_active":"2023-12-11T19:31:53.986769+0000","last_peered":"2023-12-11T19:31:53.986769+0000","last_clean":"2023-12-11T19:31:53.986769+0000","last_became_active":"2023-12-11T19:31:52.272088+0000","last_became_peered":"2023-12-11T19:31:52.272088+0000","last_unstale":"2023-12-11T19:31:53.986769+0000","last_undegraded":"2023-12-11T19:31:53.986769+0000","last_fullsized":"2023-12-11T19:31:53.986769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:37:03.906666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977003+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:31:53.977003+0000","last_peered":"2023-12-11T19:31:53.977003+0000","last_clean":"2023-12-11T19:31:53.977003+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:31:53.977003+0000","last_undegraded":"2023-12-11T19:31:53.977003+0000","last_fullsized":"2023-12-11T19:31:53.977003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.977434+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:31:53.977434+0000","last_peered":"2023-12-11T19:31:53.977434+0000","last_clean":"2023-12-11T19:31:53.977434+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:31:53.977434+0000","last_undegraded":"2023-12-11T19:31:53.977434+0000","last_fullsized":"2023-12-11T19:31:53.977434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.978193+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:31:53.978193+0000","last_peered":"2023-12-11T19:31:53.978193+0000","last_clean":"2023-12-11T19:31:53.978193+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:31:53.978193+0000","last_undegraded":"2023-12-11T19:31:53.978193+0000","last_fullsized":"2023-12-11T19:31:53.978193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.264460+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:31:53.264460+0000","last_peered":"2023-12-11T19:31:53.264460+0000","last_clean":"2023-12-11T19:31:53.264460+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:31:53.264460+0000","last_undegraded":"2023-12-11T19:31:53.264460+0000","last_fullsized":"2023-12-11T19:31:53.264460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976839+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:31:53.976839+0000","last_peered":"2023-12-11T19:31:53.976839+0000","last_clean":"2023-12-11T19:31:53.976839+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:31:53.976839+0000","last_undegraded":"2023-12-11T19:31:53.976839+0000","last_fullsized":"2023-12-11T19:31:53.976839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.263840+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:31:53.263840+0000","last_peered":"2023-12-11T19:31:53.263840+0000","last_clean":"2023-12-11T19:31:53.263840+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:31:53.263840+0000","last_undegraded":"2023-12-11T19:31:53.263840+0000","last_fullsized":"2023-12-11T19:31:53.263840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976297+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:31:53.976297+0000","last_peered":"2023-12-11T19:31:53.976297+0000","last_clean":"2023-12-11T19:31:53.976297+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:31:53.976297+0000","last_undegraded":"2023-12-11T19:31:53.976297+0000","last_fullsized":"2023-12-11T19:31:53.976297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976848+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:31:53.976848+0000","last_peered":"2023-12-11T19:31:53.976848+0000","last_clean":"2023-12-11T19:31:53.976848+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:31:53.976848+0000","last_undegraded":"2023-12-11T19:31:53.976848+0000","last_fullsized":"2023-12-11T19:31:53.976848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T19:31:53.976552+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:31:53.976552+0000","last_peered":"2023-12-11T19:31:53.976552+0000","last_clean":"2023-12-11T19:31:53.976552+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:31:53.976552+0000","last_undegraded":"2023-12-11T19:31:53.976552+0000","last_fullsized":"2023-12-11T19:31:53.976552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43600,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":29,"num_read_kb":56,"num_write":167,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":75414,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":441954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1071707,"data_compressed":38035,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.71499999999999997}]},{"osd":2,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.496}]},{"osd":4,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.439},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.85999999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":442280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.504}]},{"osd":1,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.72299999999999998}]},{"osd":7,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.74399999999999999}]},{"osd":8,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.78500000000000003}]}]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":441998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.39400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.46500000000000002,"5min":0.373,"15min":0.373},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.72299999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.5}]},{"osd":3,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70599999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.62}]},{"osd":4,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":0.754},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.73599999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81100000000000005,"15min":0.80800000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.90400000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.92600000000000005,"15min":0.92900000000000005},"min":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76300000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.94499999999999995}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.88200000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1033873,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85099999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.39000000000000001}]},{"osd":2,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98399999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.101,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.046,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.63200000000000001,"15min":0.64500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84599999999999997,"5min":0.874,"15min":0.874},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.03,"5min":1.238,"15min":1.238},"last":0.76400000000000001}]},{"osd":5,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.77400000000000002,"15min":0.78400000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97999999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.224,"5min":1.28,"15min":1.28},"last":0.45900000000000002}]},{"osd":6,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86199999999999999,"15min":0.876},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1299999999999999,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1160000000000001,"5min":1.359,"15min":1.359},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.66600000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.63},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.77000000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.274,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52600000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.751},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.73099999999999998}]},{"osd":10,"last update":"Mon Dec 11 19:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.64200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":441372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.31,"15min":0.31},"max":{"1min":0.68400000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.752}]},{"osd":2,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92500000000000004,"5min":1.284,"15min":1.284},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95599999999999996,"5min":1.228,"15min":1.228},"last":0.77700000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.016,"15min":1.016},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.119,"5min":1.157,"15min":1.157},"last":0.89200000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82399999999999995,"15min":0.81200000000000006},"min":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88600000000000001,"15min":0.92200000000000004},"min":{"1min":0.65400000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0349999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.82299999999999995,"15min":0.81799999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.873,"15min":0.86599999999999999},"min":{"1min":0.63700000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.92600000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:31:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.70999999999999996,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.204,"5min":1.212,"15min":1.212},"last":1.02},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.878,"15min":0.88300000000000001},"min":{"1min":0.64000000000000001,"5min":0.621,"15min":0.621},"max":{"1min":1.135,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.97599999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.081}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":442711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60599999999999998,"15min":0.54600000000000004},"min":{"1min":0.40200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.64200000000000002,"15min":0.56599999999999995},"min":{"1min":0.42199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0469999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.77700000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.74299999999999999,"15min":0.70199999999999996},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.1180000000000003,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0349999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Dec 11 19:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.129,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.52400000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0229999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.85199999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.60399999999999998,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Dec 11 19:31:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.502,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93799999999999994,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.81599999999999995,"15min":0.88200000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92200000000000004,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.63,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.871},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.05,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93000000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:31:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.438,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80300000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.78700000000000003},"min":{"1min":0.69299999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.96599999999999997}]},{"osd":9,"last update":"Mon Dec 11 19:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.98899999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.93999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":443438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.80400000000000005,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.05,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.64600000000000002},"min":{"1min":0.48899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.99399999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.502}]},{"osd":2,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.68899999999999995,"15min":0.627},"min":{"1min":0.50800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0960000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.67400000000000004,"15min":0.65300000000000002},"min":{"1min":0.51000000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Dec 11 19:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.46100000000000002},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77200000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":1.004,"15min":1.004},"last":0.46100000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56299999999999994}]},{"osd":5,"last update":"Mon Dec 11 19:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.65600000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.036,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.224,"5min":1.819,"15min":1.819},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Dec 11 19:31:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.64000000000000001}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.71299999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.746},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.89600000000000002},"min":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.753}]},{"osd":9,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.77800000000000002}]},{"osd":10,"last update":"Mon Dec 11 19:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":444956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80500000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82099999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91100000000000003,"5min":1.115,"15min":1.115},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.877}]},{"osd":3,"last update":"Mon Dec 11 19:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85199999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.497,"5min":0.433,"15min":0.433},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66300000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.58999999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.745}]},{"osd":5,"last update":"Mon Dec 11 19:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.41299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82699999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.79700000000000004}]},{"osd":9,"last update":"Mon Dec 11 19:31:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93400000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.95399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":441717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.434,"15min":0.39400000000000002},"min":{"1min":0.40200000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.42499999999999999},"min":{"1min":0.378,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.755,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42099999999999999}]},{"osd":1,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.626,"15min":0.59599999999999997},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.504},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.52500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.68600000000000005,"15min":0.624},"min":{"1min":0.58599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.751},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68400000000000005,"15min":0.63400000000000001},"min":{"1min":0.53500000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.054,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.75600000000000001,"15min":0.69999999999999996},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.105,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.70999999999999996}]},{"osd":6,"last update":"Mon Dec 11 19:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.83399999999999996,"15min":0.81699999999999995},"min":{"1min":0.60799999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.67900000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.81299999999999994}]},{"osd":10,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.33500000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.77100000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93999999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71699999999999997},"min":{"1min":0.497,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95899999999999996,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.45700000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:31:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70999999999999996,"15min":0.68799999999999994},"min":{"1min":0.55100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97499999999999998,"5min":1.075,"15min":1.075},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Dec 11 19:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.498},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.80300000000000005,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.80100000000000005,"15min":0.78600000000000003},"min":{"1min":0.61499999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.75600000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67100000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1034949,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.624},"min":{"1min":0.44700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.622},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.51500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81999999999999995,"5min":1.177,"15min":1.177},"last":0.66300000000000003}]},{"osd":1,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.48699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98999999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79900000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.47699999999999998,"15min":0.45800000000000002},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.75},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.50700000000000001},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.55500000000000005}]},{"osd":3,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.504},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94999999999999996,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.749,"15min":0.748},"min":{"1min":0.46000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95599999999999996,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2110000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76300000000000001,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.746},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.248,"5min":1.597,"15min":1.597},"last":0.59999999999999998}]},{"osd":7,"last update":"Mon Dec 11 19:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.749,"15min":0.73999999999999999},"min":{"1min":0.56100000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85899999999999999}]},{"osd":8,"last update":"Mon Dec 11 19:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.83099999999999996,"15min":0.82899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.129,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.75600000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":21337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":39844,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:32:01.574 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-11T19:32:01.575 INFO:tasks.cephadm:Setting up client nodes... 2023-12-11T19:32:01.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-11T19:32:01.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:01 smithi066 ceph-mon[97920]: pgmap v311: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-11T19:32:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:01 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1084793586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:01.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:01 smithi066 ceph-mon[97920]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:01.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:01 smithi070 ceph-mon[104044]: pgmap v311: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-11T19:32:01.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:01 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1084793586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:01.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:01 smithi070 ceph-mon[104044]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:02.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:01 smithi167 ceph-mon[102750]: pgmap v311: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-11T19:32:02.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:01 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1084793586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:02.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:01 smithi167 ceph-mon[102750]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:02.393 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:02.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:02 smithi066 ceph-mon[97920]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:02.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:02 smithi070 ceph-mon[104044]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:03.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:02 smithi167 ceph-mon[102750]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:03.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:03 smithi066 ceph-mon[97920]: pgmap v312: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-11T19:32:03.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:03 smithi070 ceph-mon[104044]: pgmap v312: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-11T19:32:03.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 -- 172.21.15.66:0/3379928932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01027e0 msgr2=0x7f5ea0102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:03.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 --2- 172.21.15.66:0/3379928932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01027e0 0x7f5ea0102c40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5e900099c0 tx=0x7f5e90009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 -- 172.21.15.66:0/3379928932 shutdown_connections 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 --2- 172.21.15.66:0/3379928932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea0103180 0x7f5ea0109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 --2- 172.21.15.66:0/3379928932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01015e0 0x7f5ea0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 --2- 172.21.15.66:0/3379928932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01027e0 0x7f5ea0102c40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 -- 172.21.15.66:0/3379928932 >> 172.21.15.66:0/3379928932 conn(0x7f5ea00fcb80 msgr2=0x7f5ea00fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 -- 172.21.15.66:0/3379928932 shutdown_connections 2023-12-11T19:32:03.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.950+0000 7f5ea7342700 1 -- 172.21.15.66:0/3379928932 wait complete. 2023-12-11T19:32:03.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.951+0000 7f5ea7342700 1 Processor -- start 2023-12-11T19:32:03.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 -- start start 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 0x7f5ea0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01027e0 0x7f5ea0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea0103180 0x7f5ea01a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01a11d0 con 0x7f5ea01015e0 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01a1310 con 0x7f5ea0103180 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea7342700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01a14a0 con 0x7f5ea01027e0 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea50de700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 0x7f5ea0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea50de700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 0x7f5ea0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34258/0 (socket says 172.21.15.66:34258) 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea50de700 1 -- 172.21.15.66:0/2992214786 learned_addr learned my addr 172.21.15.66:0/2992214786 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.952+0000 7f5ea58df700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea0103180 0x7f5ea01a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea48dd700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01027e0 0x7f5ea0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea50de700 1 -- 172.21.15.66:0/2992214786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01027e0 msgr2=0x7f5ea0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea50de700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01027e0 0x7f5ea0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea50de700 1 -- 172.21.15.66:0/2992214786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea0103180 msgr2=0x7f5ea01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea50de700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea0103180 0x7f5ea01a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea50de700 1 -- 172.21.15.66:0/2992214786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900095f0 con 0x7f5ea01015e0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea48dd700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01027e0 0x7f5ea0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea50de700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 0x7f5ea0199690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00e9a0 tx=0x7f5e9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c009790 con 0x7f5ea01015e0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea01a1780 con 0x7f5ea01015e0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e9c0044b0 con 0x7f5ea01015e0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.953+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c003c60 con 0x7f5ea01015e0 2023-12-11T19:32:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.954+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea01a1cd0 con 0x7f5ea01015e0 2023-12-11T19:32:03.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.955+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea004f290 con 0x7f5ea01015e0 2023-12-11T19:32:03.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.957+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5e9c003dc0 con 0x7f5ea01015e0 2023-12-11T19:32:03.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.958+0000 7f5e967fc700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e8c0a1510 0x7f5e8c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:03.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.958+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5e9c014070 con 0x7f5ea01015e0 2023-12-11T19:32:03.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.958+0000 7f5ea48dd700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e8c0a1510 0x7f5e8c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:03.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.959+0000 7f5ea48dd700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e8c0a1510 0x7f5e8c0a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000b580 tx=0x7f5e9001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:03.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:03.962+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e9c096680 con 0x7f5ea01015e0 2023-12-11T19:32:04.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:03 smithi167 ceph-mon[102750]: pgmap v312: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-11T19:32:04.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.257+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5ea010cce0 con 0x7f5ea01015e0 2023-12-11T19:32:04.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.259+0000 7f5e967fc700 1 -- 172.21.15.66:0/2992214786 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5e9c099ca0 con 0x7f5ea01015e0 2023-12-11T19:32:04.260 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2023-12-11T19:32:04.260 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQC0Y3dlR9xhDxAAF6z/VpoICxOcDxikibyqJw== 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e8c0a1510 msgr2=0x7f5e8c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e8c0a1510 0x7f5e8c0a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000b580 tx=0x7f5e9001a040 comp rx=0 tx=0).stop 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 msgr2=0x7f5ea0199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 0x7f5ea0199690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00e9a0 tx=0x7f5e9c00ecb0 comp rx=0 tx=0).stop 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 shutdown_connections 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e8c0a1510 0x7f5e8c0a39d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ea0103180 0x7f5ea01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:04.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea01015e0 0x7f5ea0199690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:04.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 --2- 172.21.15.66:0/2992214786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea01027e0 0x7f5ea0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:04.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 >> 172.21.15.66:0/2992214786 conn(0x7f5ea00fcb80 msgr2=0x7f5ea0108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:04.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 shutdown_connections 2023-12-11T19:32:04.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:04.262+0000 7f5ea7342700 1 -- 172.21.15.66:0/2992214786 wait complete. 2023-12-11T19:32:04.691 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:32:04.692 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-11T19:32:04.692 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-11T19:32:04.798 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-11T19:32:04.798 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-11T19:32:04.798 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph mgr dump --format=json 2023-12-11T19:32:04.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:04 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2992214786' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-11T19:32:04.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:04 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2992214786' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-11T19:32:04.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:04 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2992214786' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-11T19:32:04.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:04 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2992214786' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-11T19:32:05.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:04 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2992214786' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-11T19:32:05.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:04 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2992214786' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-11T19:32:05.604 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:05.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:05 smithi066 ceph-mon[97920]: pgmap v313: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.4 KiB/s wr, 26 op/s 2023-12-11T19:32:05.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:05 smithi070 ceph-mon[104044]: pgmap v313: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.4 KiB/s wr, 26 op/s 2023-12-11T19:32:06.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:05 smithi167 ceph-mon[102750]: pgmap v313: 97 pgs: 97 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.4 KiB/s wr, 26 op/s 2023-12-11T19:32:07.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 -- 172.21.15.66:0/2682190463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 msgr2=0x7fb560107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:07.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 --2- 172.21.15.66:0/2682190463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560107ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009990 tx=0x7fb55c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 -- 172.21.15.66:0/2682190463 shutdown_connections 2023-12-11T19:32:07.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 --2- 172.21.15.66:0/2682190463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb560102040 0x7fb56010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 --2- 172.21.15.66:0/2682190463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560107ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 --2- 172.21.15.66:0/2682190463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560101680 0x7fb560101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 -- 172.21.15.66:0/2682190463 >> 172.21.15.66:0/2682190463 conn(0x7fb5600787c0 msgr2=0x7fb560078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:07.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 -- 172.21.15.66:0/2682190463 shutdown_connections 2023-12-11T19:32:07.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.131+0000 7fb56809f700 1 -- 172.21.15.66:0/2682190463 wait complete. 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.132+0000 7fb56809f700 1 Processor -- start 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 -- start start 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb560101680 0x7fb56019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560102040 0x7fb56019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560197e00 con 0x7fb5601076f0 2023-12-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560197f70 con 0x7fb560102040 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56809f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560198130 con 0x7fb560101680 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56663c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56663c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34274/0 (socket says 172.21.15.66:34274) 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56663c700 1 -- 172.21.15.66:0/1442898146 learned_addr learned my addr 172.21.15.66:0/1442898146 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.133+0000 7fb56563a700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560102040 0x7fb56019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb565e3b700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb560101680 0x7fb56019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56663c700 1 -- 172.21.15.66:0/1442898146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb560101680 msgr2=0x7fb56019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56663c700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb560101680 0x7fb56019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56663c700 1 -- 172.21.15.66:0/1442898146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560102040 msgr2=0x7fb56019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56663c700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560102040 0x7fb56019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56663c700 1 -- 172.21.15.66:0/1442898146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0095f0 con 0x7fb5601076f0 2023-12-11T19:32:07.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56663c700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560197790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb55800efc0 tx=0x7fb55800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:07.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558018070 con 0x7fb5601076f0 2023-12-11T19:32:07.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55800f040 con 0x7fb5601076f0 2023-12-11T19:32:07.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5601983b0 con 0x7fb5601076f0 2023-12-11T19:32:07.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55800cbb0 con 0x7fb5601076f0 2023-12-11T19:32:07.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.134+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a4640 con 0x7fb5601076f0 2023-12-11T19:32:07.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.136+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb560109b60 con 0x7fb5601076f0 2023-12-11T19:32:07.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.138+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fb55800cd90 con 0x7fb5601076f0 2023-12-11T19:32:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.139+0000 7fb556ffd700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb54c0a1590 0x7fb54c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.139+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb5580cc710 con 0x7fb5601076f0 2023-12-11T19:32:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.139+0000 7fb565e3b700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb54c0a1590 0x7fb54c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:07.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.140+0000 7fb565e3b700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb54c0a1590 0x7fb54c0a3a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006010 tx=0x7fb55c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:07.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.144+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb558095d50 con 0x7fb5601076f0 2023-12-11T19:32:07.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.416+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb560102d10 con 0x7fb5601076f0 2023-12-11T19:32:07.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.422+0000 7fb556ffd700 1 -- 172.21.15.66:0/1442898146 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260406 (secure 0 0 0) 0x7fb558099370 con 0x7fb5601076f0 2023-12-11T19:32:07.429 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.426+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb54c0a1590 msgr2=0x7fb54c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.426+0000 7fb56809f700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb54c0a1590 0x7fb54c0a3a50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006010 tx=0x7fb55c00b260 comp rx=0 tx=0).stop 2023-12-11T19:32:07.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.426+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 msgr2=0x7fb560197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.426+0000 7fb56809f700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560197790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb55800efc0 tx=0x7fb55800c3a0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 shutdown_connections 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb54c0a1590 0x7fb54c0a3a50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5601076f0 0x7fb560197790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb560101680 0x7fb56019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 --2- 172.21.15.66:0/1442898146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb560102040 0x7fb56019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 >> 172.21.15.66:0/1442898146 conn(0x7fb5600787c0 msgr2=0x7fb5601004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 shutdown_connections 2023-12-11T19:32:07.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:07.427+0000 7fb56809f700 1 -- 172.21.15.66:0/1442898146 wait complete. 2023-12-11T19:32:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:07 smithi066 ceph-mon[97920]: pgmap v314: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 25 op/s 2023-12-11T19:32:07.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:07 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1442898146' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-11T19:32:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:07 smithi070 ceph-mon[104044]: pgmap v314: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 25 op/s 2023-12-11T19:32:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:07 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1442898146' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-11T19:32:07.899 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":775613503},{"type":"v1","addr":"172.21.15.66:6801","nonce":775613503}]},"active_addr":"172.21.15.66:6801/775613503","active_change":"2023-12-11T19:24:10.191074+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14217,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.66:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":188604032}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":980393415}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3400214772}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":2264173503}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1655052441}]}]} 2023-12-11T19:32:07.904 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-11T19:32:07.904 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-11T19:32:07.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-11T19:32:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:07 smithi167 ceph-mon[102750]: pgmap v314: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 25 op/s 2023-12-11T19:32:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:07 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1442898146' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-11T19:32:08.701 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:09.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:09 smithi070 ceph-mon[104044]: pgmap v315: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-12-11T19:32:09.956 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:09 smithi066 ceph-mon[97920]: pgmap v315: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-12-11T19:32:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:09 smithi167 ceph-mon[102750]: pgmap v315: 97 pgs: 97 active+clean; 625 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.8 KiB/s wr, 23 op/s 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 -- 172.21.15.66:0/3043048747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804109820 msgr2=0x7fa804109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 --2- 172.21.15.66:0/3043048747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804109820 0x7fa804109c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa800009990 tx=0x7fa800009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 -- 172.21.15.66:0/3043048747 shutdown_connections 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 --2- 172.21.15.66:0/3043048747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 0x7fa8041106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 --2- 172.21.15.66:0/3043048747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804109820 0x7fa804109c00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 --2- 172.21.15.66:0/3043048747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804103820 0x7fa804103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 -- 172.21.15.66:0/3043048747 >> 172.21.15.66:0/3043048747 conn(0x7fa8040ff370 msgr2=0x7fa804101790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 -- 172.21.15.66:0/3043048747 shutdown_connections 2023-12-11T19:32:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.222+0000 7fa80b524700 1 -- 172.21.15.66:0/3043048747 wait complete. 2023-12-11T19:32:10.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.224+0000 7fa80b524700 1 Processor -- start 2023-12-11T19:32:10.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.224+0000 7fa80b524700 1 -- start start 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.224+0000 7fa80b524700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 0x7fa80419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.224+0000 7fa80b524700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 0x7fa80419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.224+0000 7fa80b524700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804109820 0x7fa8041999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa80b524700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80419a030 con 0x7fa804103820 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa80b524700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80419a1a0 con 0x7fa804109820 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa80b524700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80419a360 con 0x7fa8041041e0 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 0x7fa80419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 0x7fa80419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34298/0 (socket says 172.21.15.66:34298) 2023-12-11T19:32:10.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 -- 172.21.15.66:0/836044290 learned_addr learned my addr 172.21.15.66:0/836044290 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa809ac1700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804109820 0x7fa8041999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa808abf700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 0x7fa80419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 -- 172.21.15.66:0/836044290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 msgr2=0x7fa80419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 0x7fa80419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 -- 172.21.15.66:0/836044290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804109820 msgr2=0x7fa8041999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804109820 0x7fa8041999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 -- 172.21.15.66:0/836044290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000095f0 con 0x7fa804103820 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa808abf700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 0x7fa80419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa809ac1700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804109820 0x7fa8041999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.225+0000 7fa8092c0700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 0x7fa80419f890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8000098e0 tx=0x7fa8000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.226+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80001d070 con 0x7fa804103820 2023-12-11T19:32:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.226+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa80000fc00 con 0x7fa804103820 2023-12-11T19:32:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.226+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80419a5e0 con 0x7fa804103820 2023-12-11T19:32:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.226+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800022cd0 con 0x7fa804103820 2023-12-11T19:32:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.226+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8041a6880 con 0x7fa804103820 2023-12-11T19:32:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.227+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80410be20 con 0x7fa804103820 2023-12-11T19:32:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.229+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa800022470 con 0x7fa804103820 2023-12-11T19:32:10.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.232+0000 7fa7fa7fc700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa7f00a1590 0x7fa7f00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:10.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.232+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8000ce180 con 0x7fa804103820 2023-12-11T19:32:10.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.232+0000 7fa808abf700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa7f00a1590 0x7fa7f00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:10.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.233+0000 7fa808abf700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa7f00a1590 0x7fa7f00a3a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005fd0 tx=0x7fa7f4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:10.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.236+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa8000977c0 con 0x7fa804103820 2023-12-11T19:32:10.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.450+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa804104f60 con 0x7fa804103820 2023-12-11T19:32:10.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.451+0000 7fa7fa7fc700 1 -- 172.21.15.66:0/836044290 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22011 (secure 0 0 0) 0x7fa80009ade0 con 0x7fa804103820 2023-12-11T19:32:10.452 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:10.453 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":86,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:31:53.249050+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa7f00a1590 msgr2=0x7fa7f00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa7f00a1590 0x7fa7f00a3a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005fd0 tx=0x7fa7f4005e20 comp rx=0 tx=0).stop 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 msgr2=0x7fa80419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 0x7fa80419f890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8000098e0 tx=0x7fa8000057f0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 shutdown_connections 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa7f00a1590 0x7fa7f00a3a50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa804109820 0x7fa8041999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa804103820 0x7fa80419f890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 --2- 172.21.15.66:0/836044290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8041041e0 0x7fa80419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 >> 172.21.15.66:0/836044290 conn(0x7fa8040ff370 msgr2=0x7fa8040ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 shutdown_connections 2023-12-11T19:32:10.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:10.454+0000 7fa80b524700 1 -- 172.21.15.66:0/836044290 wait complete. 2023-12-11T19:32:10.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-12-11T19:32:10.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:10 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/836044290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-12-11T19:32:10.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-12-11T19:32:10.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:10 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/836044290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:11.001 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-11T19:32:11.001 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-11T19:32:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-11T19:32:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-12-11T19:32:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-12-11T19:32:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-11T19:32:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-12-11T19:32:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-12-11T19:32:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-11T19:32:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-12-11T19:32:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-12-11T19:32:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:10 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/836044290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:11.797 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:11.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 25 op/s 2023-12-11T19:32:11.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-11T19:32:11.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-12-11T19:32:11.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-12-11T19:32:11.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-11T19:32:11.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-12-11T19:32:11.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-12-11T19:32:11.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-11T19:32:11.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-12-11T19:32:11.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-12-11T19:32:11.861 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:11 smithi070 ceph-mon[104044]: osdmap e87: 12 total, 12 up, 12 in 2023-12-11T19:32:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 25 op/s 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-12-11T19:32:12.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:11 smithi167 ceph-mon[102750]: osdmap e87: 12 total, 12 up, 12 in 2023-12-11T19:32:12.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 25 op/s 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-12-11T19:32:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:11 smithi066 ceph-mon[97920]: osdmap e87: 12 total, 12 up, 12 in 2023-12-11T19:32:12.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:12 smithi066 ceph-mon[97920]: osdmap e88: 12 total, 12 up, 12 in 2023-12-11T19:32:12.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:12 smithi070 ceph-mon[104044]: osdmap e88: 12 total, 12 up, 12 in 2023-12-11T19:32:13.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:12 smithi167 ceph-mon[102750]: osdmap e88: 12 total, 12 up, 12 in 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 -- 172.21.15.66:0/1511171408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b141076f0 msgr2=0x7f3b14107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 --2- 172.21.15.66:0/1511171408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b141076f0 0x7f3b14107ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009990 tx=0x7f3b10009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 -- 172.21.15.66:0/1511171408 shutdown_connections 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 --2- 172.21.15.66:0/1511171408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b14102040 0x7f3b1410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 --2- 172.21.15.66:0/1511171408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b141076f0 0x7f3b14107ad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 --2- 172.21.15.66:0/1511171408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b14101680 0x7f3b14101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.330+0000 7f3b1c096700 1 -- 172.21.15.66:0/1511171408 >> 172.21.15.66:0/1511171408 conn(0x7f3b140787c0 msgr2=0x7f3b14078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:13.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.331+0000 7f3b1c096700 1 -- 172.21.15.66:0/1511171408 shutdown_connections 2023-12-11T19:32:13.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.331+0000 7f3b1c096700 1 -- 172.21.15.66:0/1511171408 wait complete. 2023-12-11T19:32:13.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 Processor -- start 2023-12-11T19:32:13.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 -- start start 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b14101680 0x7f3b1419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 0x7f3b1419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b141076f0 0x7f3b14199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14199f70 con 0x7f3b14102040 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1419a0e0 con 0x7f3b141076f0 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.332+0000 7f3b1c096700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1419a2a0 con 0x7f3b14101680 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 0x7f3b1419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 0x7f3b1419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45288/0 (socket says 172.21.15.66:45288) 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 -- 172.21.15.66:0/691711937 learned_addr learned my addr 172.21.15.66:0/691711937 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19e32700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b14101680 0x7f3b1419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b1a633700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b141076f0 0x7f3b14199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 -- 172.21.15.66:0/691711937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b14101680 msgr2=0x7f3b1419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:13.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b14101680 0x7f3b1419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 -- 172.21.15.66:0/691711937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b141076f0 msgr2=0x7f3b14199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b141076f0 0x7f3b14199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.333+0000 7f3b19631700 1 -- 172.21.15.66:0/691711937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100095f0 con 0x7f3b14102040 2023-12-11T19:32:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.334+0000 7f3b19631700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 0x7f3b1419fd10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400b530 tx=0x7f3b0400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.334+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04011830 con 0x7f3b14102040 2023-12-11T19:32:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.334+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b04011e20 con 0x7f3b14102040 2023-12-11T19:32:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.334+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1419a580 con 0x7f3b14102040 2023-12-11T19:32:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.334+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04010db0 con 0x7f3b14102040 2023-12-11T19:32:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.334+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b141a67c0 con 0x7f3b14102040 2023-12-11T19:32:13.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.335+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b14104940 con 0x7f3b14102040 2023-12-11T19:32:13.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.337+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3b04011990 con 0x7f3b14102040 2023-12-11T19:32:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.338+0000 7f3b0affd700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b000a1590 0x7f3b000a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.339+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b040cc6f0 con 0x7f3b14102040 2023-12-11T19:32:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.339+0000 7f3b19e32700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b000a1590 0x7f3b000a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.339+0000 7f3b19e32700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b000a1590 0x7f3b000a3a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b5c0 tx=0x7f3b10005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:13.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.343+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3b04095cc0 con 0x7f3b14102040 2023-12-11T19:32:13.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.555+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b14102d10 con 0x7f3b14102040 2023-12-11T19:32:13.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.556+0000 7f3b0affd700 1 -- 172.21.15.66:0/691711937 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f3b040992e0 con 0x7f3b14102040 2023-12-11T19:32:13.557 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:13.558 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b000a1590 msgr2=0x7f3b000a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b000a1590 0x7f3b000a3a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b5c0 tx=0x7f3b10005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 msgr2=0x7f3b1419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 0x7f3b1419fd10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400b530 tx=0x7f3b0400b840 comp rx=0 tx=0).stop 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 shutdown_connections 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b000a1590 0x7f3b000a3a50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b141076f0 0x7f3b14199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b14101680 0x7f3b1419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 --2- 172.21.15.66:0/691711937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b14102040 0x7f3b1419fd10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 >> 172.21.15.66:0/691711937 conn(0x7f3b140787c0 msgr2=0x7f3b1410ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 shutdown_connections 2023-12-11T19:32:13.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:13.559+0000 7f3b1c096700 1 -- 172.21.15.66:0/691711937 wait complete. 2023-12-11T19:32:13.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:13 smithi070 ceph-mon[104044]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-12-11T19:32:13.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:13 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/691711937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:13.986 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:13 smithi066 ceph-mon[97920]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-12-11T19:32:13.986 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:13 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/691711937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:13.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-11T19:32:13.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-11T19:32:13.990 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-11T19:32:13.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-11T19:32:13.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-11T19:32:13.992 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-11T19:32:13.992 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-11T19:32:13.992 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-11T19:32:13.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-11T19:32:13.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-11T19:32:13.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-11T19:32:13.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-11T19:32:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:13 smithi167 ceph-mon[102750]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-12-11T19:32:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:13 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/691711937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:15.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:15 smithi070 ceph-mon[104044]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 141 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:16.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:15 smithi167 ceph-mon[102750]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 141 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:16.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:15 smithi066 ceph-mon[97920]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 141 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:16.283 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.321 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.372 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.447 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.448 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.744 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.825 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:16.874 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:17.131 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:17.375 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:17.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:17 smithi066 ceph-mon[97920]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 287 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:17.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:17 smithi070 ceph-mon[104044]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 287 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:18.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:17 smithi167 ceph-mon[102750]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 287 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:19.862 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:19 smithi070 ceph-mon[104044]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 287 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:19.863 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:19 smithi066 ceph-mon[97920]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 287 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:20.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:19 smithi167 ceph-mon[102750]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 287 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:21.853 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:21 smithi066 ceph-mon[97920]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 526 B/s wr, 8 op/s; 237 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:21 smithi167 ceph-mon[102750]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 526 B/s wr, 8 op/s; 237 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:22.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:21 smithi070 ceph-mon[104044]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 526 B/s wr, 8 op/s; 237 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 -- 172.21.15.66:0/1647291284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe881076f0 msgr2=0x7fbe88107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/1647291284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe881076f0 0x7fbe88107ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009990 tx=0x7fbe78009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 -- 172.21.15.66:0/1647291284 shutdown_connections 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/1647291284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe88102040 0x7fbe8810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/1647291284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe881076f0 0x7fbe88107ad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/1647291284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe88101680 0x7fbe88101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 -- 172.21.15.66:0/1647291284 >> 172.21.15.66:0/1647291284 conn(0x7fbe880787c0 msgr2=0x7fbe88078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 -- 172.21.15.66:0/1647291284 shutdown_connections 2023-12-11T19:32:22.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.480+0000 7fbe8dd23700 1 -- 172.21.15.66:0/1647291284 wait complete. 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.481+0000 7fbe8dd23700 1 Processor -- start 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.481+0000 7fbe8dd23700 1 -- start start 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe8dd23700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe88101680 0x7fbe8819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe8dd23700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe88102040 0x7fbe8819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe8dd23700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe881076f0 0x7fbe88199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe8dd23700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88199f60 con 0x7fbe88101680 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe8dd23700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8819a0d0 con 0x7fbe881076f0 2023-12-11T19:32:22.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe8dd23700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8819a290 con 0x7fbe88102040 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe877fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe88101680 0x7fbe8819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe877fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe88101680 0x7fbe8819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60654/0 (socket says 172.21.15.66:60654) 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.482+0000 7fbe877fe700 1 -- 172.21.15.66:0/2991920721 learned_addr learned my addr 172.21.15.66:0/2991920721 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe87fff700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe881076f0 0x7fbe88199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe86ffd700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe88102040 0x7fbe8819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe877fe700 1 -- 172.21.15.66:0/2991920721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe88102040 msgr2=0x7fbe8819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe877fe700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe88102040 0x7fbe8819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe877fe700 1 -- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe881076f0 msgr2=0x7fbe88199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe877fe700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe881076f0 0x7fbe88199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe877fe700 1 -- 172.21.15.66:0/2991920721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780095f0 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe86ffd700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe88102040 0x7fbe8819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe877fe700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe88101680 0x7fbe8819f7c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe780036c0 tx=0x7fbe78005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7801d070 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8819a510 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7800fc00 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.483+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78022c40 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.484+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe881a67b0 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.484+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe88111690 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.487+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbe78022470 con 0x7fbe88101680 2023-12-11T19:32:22.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.488+0000 7fbe84ff9700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1510 0x7fbe700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.488+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe780ceab0 con 0x7fbe88101680 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.489+0000 7fbe86ffd700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1510 0x7fbe700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.489+0000 7fbe84ff9700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] conn(0x7fbe700ab0d0 0x7fbe700ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.489+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 --> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe700adc40 con 0x7fbe700ab0d0 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.489+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe780cee20 con 0x7fbe88101680 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.489+0000 7fbe86ffd700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1510 0x7fbe700a39d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c005fd0 tx=0x7fbe7c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.489+0000 7fbe87fff700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] conn(0x7fbe700ab0d0 0x7fbe700ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.490+0000 7fbe87fff700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] conn(0x7fbe700ab0d0 0x7fbe700ad570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.491+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== osd.10 v2:172.21.15.70:6824/2336173720 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbe700adc40 con 0x7fbe700ab0d0 2023-12-11T19:32:22.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.509+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 --> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe88061c50 con 0x7fbe700ab0d0 2023-12-11T19:32:22.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.510+0000 7fbe84ff9700 1 -- 172.21.15.66:0/2991920721 <== osd.10 v2:172.21.15.70:6824/2336173720 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbe88061c50 con 0x7fbe700ab0d0 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] conn(0x7fbe700ab0d0 msgr2=0x7fbe700ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] conn(0x7fbe700ab0d0 0x7fbe700ad570 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1510 msgr2=0x7fbe700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1510 0x7fbe700a39d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c005fd0 tx=0x7fbe7c005ee0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe88101680 msgr2=0x7fbe8819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 --2- 172.21.15.66:0/2991920721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe88101680 0x7fbe8819f7c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe780036c0 tx=0x7fbe78005e80 comp rx=0 tx=0).stop 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe87fff700 1 -- 172.21.15.66:0/2991920721 reap_dead start 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 shutdown_connections 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 >> 172.21.15.66:0/2991920721 conn(0x7fbe880787c0 msgr2=0x7fbe8810caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:22.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 shutdown_connections 2023-12-11T19:32:22.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.511+0000 7fbe8dd23700 1 -- 172.21.15.66:0/2991920721 wait complete. 2023-12-11T19:32:22.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.671+0000 7f2f84ec8700 1 -- 172.21.15.66:0/871038215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80068f70 msgr2=0x7f2f80069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.671+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/871038215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80068f70 0x7f2f80069350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009990 tx=0x7f2f70009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 -- 172.21.15.66:0/871038215 shutdown_connections 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/871038215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8010e0e0 0x7f2f801104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/871038215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80068f70 0x7f2f80069350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/871038215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80069890 0x7f2f8010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 -- 172.21.15.66:0/871038215 >> 172.21.15.66:0/871038215 conn(0x7f2f80076570 msgr2=0x7f2f80076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 -- 172.21.15.66:0/871038215 shutdown_connections 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.672+0000 7f2f84ec8700 1 -- 172.21.15.66:0/871038215 wait complete. 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 Processor -- start 2023-12-11T19:32:22.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 -- start start 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80068f70 0x7f2f8019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80069890 0x7f2f8019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8010e0e0 0x7f2f80199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80199f80 con 0x7f2f80069890 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8019a0f0 con 0x7f2f8010e0e0 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.673+0000 7f2f84ec8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8019a2b0 con 0x7f2f80068f70 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80069890 0x7f2f8019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80068f70 0x7f2f8019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80068f70 0x7f2f8019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:59542/0 (socket says 172.21.15.66:59542) 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 -- 172.21.15.66:0/620222761 learned_addr learned my addr 172.21.15.66:0/620222761 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:22.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80069890 0x7f2f8019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60672/0 (socket says 172.21.15.66:60672) 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7ed9d700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8010e0e0 0x7f2f80199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 -- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8010e0e0 msgr2=0x7f2f80199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8010e0e0 0x7f2f80199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 -- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80069890 msgr2=0x7f2f8019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f80069890 0x7f2f8019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 -- 172.21.15.66:0/620222761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700095f0 con 0x7f2f80068f70 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7ed9d700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8010e0e0 0x7f2f80199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.674+0000 7f2f7e59c700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80068f70 0x7f2f8019f7e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f70006010 tx=0x7f2f70005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.675+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7001d070 con 0x7f2f80068f70 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.675+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f7000fbc0 con 0x7f2f80068f70 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.675+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8019a530 con 0x7f2f80068f70 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.675+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70022b50 con 0x7f2f80068f70 2023-12-11T19:32:22.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.675+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f801a67d0 con 0x7f2f80068f70 2023-12-11T19:32:22.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.676+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f80113860 con 0x7f2f80068f70 2023-12-11T19:32:22.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.678+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2f7000fd30 con 0x7f2f80068f70 2023-12-11T19:32:22.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.679+0000 7f2f6f7fe700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2f680a1590 0x7f2f680a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.679+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f700cee00 con 0x7f2f80068f70 2023-12-11T19:32:22.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.680+0000 7f2f7dd9b700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2f680a1590 0x7f2f680a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.680+0000 7f2f6f7fe700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] conn(0x7f2f680ab150 0x7f2f680ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.680+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 --> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f680adcc0 con 0x7f2f680ab150 2023-12-11T19:32:22.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.680+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f700cf170 con 0x7f2f80068f70 2023-12-11T19:32:22.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.680+0000 7f2f7dd9b700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2f680a1590 0x7f2f680a3a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2f8019aae0 tx=0x7f2f7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.680+0000 7f2f7ed9d700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] conn(0x7f2f680ab150 0x7f2f680ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.681+0000 7f2f7ed9d700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] conn(0x7f2f680ab150 0x7f2f680ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.682+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== osd.4 v2:172.21.15.70:6808/161576683 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2f680adcc0 con 0x7f2f680ab150 2023-12-11T19:32:22.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.699+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 --> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f80061c50 con 0x7f2f680ab150 2023-12-11T19:32:22.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.700+0000 7f2f6f7fe700 1 -- 172.21.15.66:0/620222761 <== osd.4 v2:172.21.15.70:6808/161576683 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2f80061c50 con 0x7f2f680ab150 2023-12-11T19:32:22.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] conn(0x7f2f680ab150 msgr2=0x7f2f680ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] conn(0x7f2f680ab150 0x7f2f680ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2f680a1590 msgr2=0x7f2f680a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2f680a1590 0x7f2f680a3a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2f8019aae0 tx=0x7f2f7400b410 comp rx=0 tx=0).stop 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80068f70 msgr2=0x7f2f8019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f84ec8700 1 --2- 172.21.15.66:0/620222761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f80068f70 0x7f2f8019f7e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f70006010 tx=0x7f2f70005720 comp rx=0 tx=0).stop 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.701+0000 7f2f7ed9d700 1 -- 172.21.15.66:0/620222761 reap_dead start 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.702+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 shutdown_connections 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.702+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 >> 172.21.15.66:0/620222761 conn(0x7f2f80076570 msgr2=0x7f2f8010f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.702+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 shutdown_connections 2023-12-11T19:32:22.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.702+0000 7f2f84ec8700 1 -- 172.21.15.66:0/620222761 wait complete. 2023-12-11T19:32:22.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 -- 172.21.15.66:0/1995622240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5748074120 msgr2=0x7f574810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 --2- 172.21.15.66:0/1995622240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5748074120 0x7f574810daf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f573c009990 tx=0x7f573c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 -- 172.21.15.66:0/1995622240 shutdown_connections 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 --2- 172.21.15.66:0/1995622240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748073800 0x7f5748073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 --2- 172.21.15.66:0/1995622240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f574810e030 0x7f5748110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 --2- 172.21.15.66:0/1995622240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5748074120 0x7f574810daf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 -- 172.21.15.66:0/1995622240 >> 172.21.15.66:0/1995622240 conn(0x7f57480787c0 msgr2=0x7f5748078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 -- 172.21.15.66:0/1995622240 shutdown_connections 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.730+0000 7f574d054700 1 -- 172.21.15.66:0/1995622240 wait complete. 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.731+0000 7f574d054700 1 Processor -- start 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.731+0000 7f574d054700 1 -- start start 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574d054700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748073800 0x7f574819f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574d054700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748074120 0x7f574819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574d054700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574810e030 0x7f5748199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574d054700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574819a130 con 0x7f574810e030 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574d054700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574819a2a0 con 0x7f5748074120 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574d054700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574819a460 con 0x7f5748073800 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574810e030 0x7f5748199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574810e030 0x7f5748199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60678/0 (socket says 172.21.15.66:60678) 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 -- 172.21.15.66:0/2889517999 learned_addr learned my addr 172.21.15.66:0/2889517999 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574659c700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748074120 0x7f574819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f5746d9d700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748073800 0x7f574819f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 -- 172.21.15.66:0/2889517999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748073800 msgr2=0x7f574819f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748073800 0x7f574819f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 -- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748074120 msgr2=0x7f574819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748074120 0x7f574819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.732+0000 7f574759e700 1 -- 172.21.15.66:0/2889517999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0095f0 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.733+0000 7f574759e700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574810e030 0x7f5748199ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f574000ab40 tx=0x7f574000ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.733+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000f070 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.733+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5740007a70 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.733+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574819a740 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.733+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740004030 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.733+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57481a6980 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.734+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57481139b0 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.736+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f574000c740 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.738+0000 7f572ffff700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57300a1590 0x7f57300a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.738+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57400cc390 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.738+0000 7f5746d9d700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57300a1590 0x7f57300a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.738+0000 7f572ffff700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] conn(0x7f57300ab150 0x7f57300ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.738+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 --> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57300adcc0 con 0x7f57300ab150 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.738+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57400cc700 con 0x7f574810e030 2023-12-11T19:32:22.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.739+0000 7f5746d9d700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57300a1590 0x7f57300a3a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5738005950 tx=0x7f57380058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.739+0000 7f574659c700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] conn(0x7f57300ab150 0x7f57300ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.739+0000 7f574659c700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] conn(0x7f57300ab150 0x7f57300ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.740+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== osd.7 v2:172.21.15.70:6816/4038002078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f57300adcc0 con 0x7f57300ab150 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.758+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 --> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5748061c50 con 0x7f57300ab150 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.758+0000 7f572ffff700 1 -- 172.21.15.66:0/2889517999 <== osd.7 v2:172.21.15.70:6816/4038002078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5748061c50 con 0x7f57300ab150 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] conn(0x7f57300ab150 msgr2=0x7f57300ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] conn(0x7f57300ab150 0x7f57300ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57300a1590 msgr2=0x7f57300a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57300a1590 0x7f57300a3a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5738005950 tx=0x7f57380058e0 comp rx=0 tx=0).stop 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574810e030 msgr2=0x7f5748199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 --2- 172.21.15.66:0/2889517999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574810e030 0x7f5748199ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f574000ab40 tx=0x7f574000ae50 comp rx=0 tx=0).stop 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574759e700 1 -- 172.21.15.66:0/2889517999 reap_dead start 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 shutdown_connections 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 >> 172.21.15.66:0/2889517999 conn(0x7f57480787c0 msgr2=0x7f57480ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 shutdown_connections 2023-12-11T19:32:22.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:22.759+0000 7f574d054700 1 -- 172.21.15.66:0/2889517999 wait complete. 2023-12-11T19:32:23.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 -- 172.21.15.66:0/1333609925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81037a0 msgr2=0x7fe7c8103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:23.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/1333609925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81037a0 0x7fe7c8103c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009990 tx=0x7fe7b4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 -- 172.21.15.66:0/1333609925 shutdown_connections 2023-12-11T19:32:23.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/1333609925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7c8104160 0x7fe7c8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/1333609925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c81097a0 0x7fe7c8109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/1333609925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81037a0 0x7fe7c8103c20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.684+0000 7fe7ceab0700 1 -- 172.21.15.66:0/1333609925 >> 172.21.15.66:0/1333609925 conn(0x7fe7c80ff250 msgr2=0x7fe7c8101670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:23.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.685+0000 7fe7ceab0700 1 -- 172.21.15.66:0/1333609925 shutdown_connections 2023-12-11T19:32:23.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.685+0000 7fe7ceab0700 1 -- 172.21.15.66:0/1333609925 wait complete. 2023-12-11T19:32:23.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 Processor -- start 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 -- start start 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7c81037a0 0x7fe7c819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c8104160 0x7fe7c819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81097a0 0x7fe7c81999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c819a050 con 0x7fe7c81097a0 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c819a1c0 con 0x7fe7c81037a0 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7ceab0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c819a380 con 0x7fe7c8104160 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.686+0000 7fe7cd04d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81097a0 0x7fe7c81999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c8104160 0x7fe7c819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:23.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7cd04d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81097a0 0x7fe7c81999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60704/0 (socket says 172.21.15.66:60704) 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7cd04d700 1 -- 172.21.15.66:0/3710936074 learned_addr learned my addr 172.21.15.66:0/3710936074 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7cc84c700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7c81037a0 0x7fe7c819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 -- 172.21.15.66:0/3710936074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7c81037a0 msgr2=0x7fe7c819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7c81037a0 0x7fe7c819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 -- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81097a0 msgr2=0x7fe7c81999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81097a0 0x7fe7c81999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 -- 172.21.15.66:0/3710936074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40095f0 con 0x7fe7c8104160 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7cd04d700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7c81097a0 0x7fe7c81999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7cc84c700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7c81037a0 0x7fe7c819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:23.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.687+0000 7fe7c7fff700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c8104160 0x7fe7c819fdf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40098e0 tx=0x7fe7b4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:23.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.688+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b401d070 con 0x7fe7c8104160 2023-12-11T19:32:23.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.688+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c819a600 con 0x7fe7c8104160 2023-12-11T19:32:23.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.688+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c81a68a0 con 0x7fe7c8104160 2023-12-11T19:32:23.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.688+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7b400fb00 con 0x7fe7c8104160 2023-12-11T19:32:23.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.688+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b402b8f0 con 0x7fe7c8104160 2023-12-11T19:32:23.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.689+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7ac000fc0 con 0x7fe7c8104160 2023-12-11T19:32:23.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.691+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe7b400fc70 con 0x7fe7c8104160 2023-12-11T19:32:23.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.692+0000 7fe7c5ffb700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe7b80a1510 0x7fe7b80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:23.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.692+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe7b40cdf80 con 0x7fe7c8104160 2023-12-11T19:32:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.692+0000 7fe7cc84c700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe7b80a1510 0x7fe7b80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.693+0000 7fe7c5ffb700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] conn(0x7fe7b80ab0d0 0x7fe7b80ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.693+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 --> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7b80adc40 con 0x7fe7b80ab0d0 2023-12-11T19:32:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.693+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b40ce2f0 con 0x7fe7c8104160 2023-12-11T19:32:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.693+0000 7fe7cd04d700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] conn(0x7fe7b80ab0d0 0x7fe7b80ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:23.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.693+0000 7fe7cc84c700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe7b80a1510 0x7fe7b80a39d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0103d0 tx=0x7fe7bc00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:23.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.694+0000 7fe7cd04d700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] conn(0x7fe7b80ab0d0 0x7fe7b80ad570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:23.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.695+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== osd.3 v2:172.21.15.66:6810/1707492505 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe7b80adc40 con 0x7fe7b80ab0d0 2023-12-11T19:32:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.713+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 --> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7ac002cb0 con 0x7fe7b80ab0d0 2023-12-11T19:32:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7c5ffb700 1 -- 172.21.15.66:0/3710936074 <== osd.3 v2:172.21.15.66:6810/1707492505 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe7ac002cb0 con 0x7fe7b80ab0d0 2023-12-11T19:32:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] conn(0x7fe7b80ab0d0 msgr2=0x7fe7b80ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:23.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] conn(0x7fe7b80ab0d0 0x7fe7b80ad570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe7b80a1510 msgr2=0x7fe7b80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe7b80a1510 0x7fe7b80a39d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0103d0 tx=0x7fe7bc00b3c0 comp rx=0 tx=0).stop 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c8104160 msgr2=0x7fe7c819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 --2- 172.21.15.66:0/3710936074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7c8104160 0x7fe7c819fdf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40098e0 tx=0x7fe7b4009580 comp rx=0 tx=0).stop 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7cd04d700 1 -- 172.21.15.66:0/3710936074 reap_dead start 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 shutdown_connections 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.714+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 >> 172.21.15.66:0/3710936074 conn(0x7fe7c80ff250 msgr2=0x7fe7c80ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.715+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 shutdown_connections 2023-12-11T19:32:23.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:23.715+0000 7fe7ceab0700 1 -- 172.21.15.66:0/3710936074 wait complete. 2023-12-11T19:32:23.867 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:23 smithi066 ceph-mon[97920]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 477 B/s wr, 9 op/s; 215 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:24.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:23 smithi167 ceph-mon[102750]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 477 B/s wr, 9 op/s; 215 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:24.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:23 smithi070 ceph-mon[104044]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 477 B/s wr, 9 op/s; 215 B/s, 0 keys/s, 1 objects/s recovering 2023-12-11T19:32:24.357 INFO:teuthology.orchestra.run.smithi066.stdout:223338299432 2023-12-11T19:32:24.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-11T19:32:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 -- 172.21.15.66:0/1350423652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167410e030 msgr2=0x7f1674110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1350423652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167410e030 0x7f1674110420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b0a0 tx=0x7f166c00b3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 -- 172.21.15.66:0/1350423652 shutdown_connections 2023-12-11T19:32:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1350423652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f167410e030 0x7f1674110420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1350423652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1674073800 0x7f1674073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1350423652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1674074120 0x7f167410daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 -- 172.21.15.66:0/1350423652 >> 172.21.15.66:0/1350423652 conn(0x7f16740787c0 msgr2=0x7f1674078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 -- 172.21.15.66:0/1350423652 shutdown_connections 2023-12-11T19:32:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.514+0000 7f1678ce4700 1 -- 172.21.15.66:0/1350423652 wait complete. 2023-12-11T19:32:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.515+0000 7f1678ce4700 1 Processor -- start 2023-12-11T19:32:24.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 -- start start 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1674073800 0x7f167419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1674074120 0x7f167419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f167410e030 0x7f1674199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419a1e0 con 0x7f1674074120 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419a350 con 0x7f1674073800 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1678ce4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419a510 con 0x7f167410e030 2023-12-11T19:32:24.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1671d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1674074120 0x7f167419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f167259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1674073800 0x7f167419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f167259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1674073800 0x7f167419fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36276/0 (socket says 172.21.15.66:36276) 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f167259c700 1 -- 172.21.15.66:0/1913895028 learned_addr learned my addr 172.21.15.66:0/1913895028 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.516+0000 7f1671d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1674074120 0x7f167419ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60732/0 (socket says 172.21.15.66:60732) 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1672d9d700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f167410e030 0x7f1674199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1671d9b700 1 -- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f167410e030 msgr2=0x7f1674199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1671d9b700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f167410e030 0x7f1674199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1671d9b700 1 -- 172.21.15.66:0/1913895028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1674073800 msgr2=0x7f167419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1671d9b700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1674073800 0x7f167419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1671d9b700 1 -- 172.21.15.66:0/1913895028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c009d00 con 0x7f1674074120 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1671d9b700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1674074120 0x7f167419ff80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f166800d950 tx=0x7f166800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668009930 con 0x7f1674074120 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1668010460 con 0x7f1674074120 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.517+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167419a7f0 con 0x7f1674074120 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.518+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800b5d0 con 0x7f1674074120 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.518+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741a6a30 con 0x7f1674074120 2023-12-11T19:32:24.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.518+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1674113920 con 0x7f1674074120 2023-12-11T19:32:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.521+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1668009a90 con 0x7f1674074120 2023-12-11T19:32:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.522+0000 7f16637fe700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f165c0a1510 0x7f165c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.523+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f16680ccda0 con 0x7f1674074120 2023-12-11T19:32:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.523+0000 7f167259c700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f165c0a1510 0x7f165c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.523+0000 7f16637fe700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] conn(0x7f165c0ab0d0 0x7f165c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.523+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 --> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f165c0adc40 con 0x7f165c0ab0d0 2023-12-11T19:32:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.523+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16680cd190 con 0x7f1674074120 2023-12-11T19:32:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.523+0000 7f167259c700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f165c0a1510 0x7f165c0a39d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1664007c90 tx=0x7f1664007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.532+0000 7f1672d9d700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] conn(0x7f165c0ab0d0 0x7f165c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.533+0000 7f1672d9d700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] conn(0x7f165c0ab0d0 0x7f165c0ad570 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:24.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.534+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== osd.8 v2:172.21.15.167:6816/2623463019 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f165c0adc40 con 0x7f165c0ab0d0 2023-12-11T19:32:24.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.552+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 --> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1674061c50 con 0x7f165c0ab0d0 2023-12-11T19:32:24.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.553+0000 7f16637fe700 1 -- 172.21.15.66:0/1913895028 <== osd.8 v2:172.21.15.167:6816/2623463019 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1674061c50 con 0x7f165c0ab0d0 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] conn(0x7f165c0ab0d0 msgr2=0x7f165c0ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] conn(0x7f165c0ab0d0 0x7f165c0ad570 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f165c0a1510 msgr2=0x7f165c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f165c0a1510 0x7f165c0a39d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1664007c90 tx=0x7f1664007480 comp rx=0 tx=0).stop 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1674074120 msgr2=0x7f167419ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 --2- 172.21.15.66:0/1913895028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1674074120 0x7f167419ff80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f166800d950 tx=0x7f166800dc60 comp rx=0 tx=0).stop 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1672d9d700 1 -- 172.21.15.66:0/1913895028 reap_dead start 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 shutdown_connections 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 >> 172.21.15.66:0/1913895028 conn(0x7f16740787c0 msgr2=0x7f16740fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 shutdown_connections 2023-12-11T19:32:24.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:24.554+0000 7f1678ce4700 1 -- 172.21.15.66:0/1913895028 wait complete. 2023-12-11T19:32:25.063 INFO:teuthology.orchestra.run.smithi066.stdout:313532612633 2023-12-11T19:32:25.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-11T19:32:25.123 INFO:teuthology.orchestra.run.smithi066.stdout:146028888122 2023-12-11T19:32:25.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-11T19:32:25.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:25 smithi066 ceph-mon[97920]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s; 191 B/s, 0 keys/s, 0 objects/s recovering 2023-12-11T19:32:25.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.589+0000 7fc9469a1700 1 -- 172.21.15.66:0/1972761593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940073d40 msgr2=0x7fc9400741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:25.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.589+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1972761593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940073d40 0x7fc9400741c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc934009990 tx=0x7fc934009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 -- 172.21.15.66:0/1972761593 shutdown_connections 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1972761593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940112350 0x7fc940114780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1972761593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc940075550 0x7fc940073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1972761593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940073d40 0x7fc9400741c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 -- 172.21.15.66:0/1972761593 >> 172.21.15.66:0/1972761593 conn(0x7fc9400787c0 msgr2=0x7fc940078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 -- 172.21.15.66:0/1972761593 shutdown_connections 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.590+0000 7fc9469a1700 1 -- 172.21.15.66:0/1972761593 wait complete. 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.591+0000 7fc9469a1700 1 Processor -- start 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.591+0000 7fc9469a1700 1 -- start start 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc9469a1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc940073d40 0x7fc94019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc9469a1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940075550 0x7fc94019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc9469a1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940112350 0x7fc9401a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc9469a1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401a2ea0 con 0x7fc940112350 2023-12-11T19:32:25.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc9469a1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401a3010 con 0x7fc940075550 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc9469a1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401a31d0 con 0x7fc940073d40 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940075550 0x7fc94019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc944f3e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940112350 0x7fc9401a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940075550 0x7fc94019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36302/0 (socket says 172.21.15.66:36302) 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 -- 172.21.15.66:0/1677385443 learned_addr learned my addr 172.21.15.66:0/1677385443 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93ffff700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc940073d40 0x7fc94019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 -- 172.21.15.66:0/1677385443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc940073d40 msgr2=0x7fc94019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc940073d40 0x7fc94019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 -- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940112350 msgr2=0x7fc9401a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc940112350 0x7fc9401a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.592+0000 7fc93f7fe700 1 -- 172.21.15.66:0/1677385443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340095f0 con 0x7fc940075550 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.593+0000 7fc93f7fe700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940075550 0x7fc94019e4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc934004760 tx=0x7fc934004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.593+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93401d070 con 0x7fc940075550 2023-12-11T19:32:25.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.593+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9401a3450 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.594+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9401a3710 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.594+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc934005a00 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.594+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934005280 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.594+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc940117d50 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.597+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc934004930 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.598+0000 7fc93d7fa700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9280a1590 0x7fc9280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.598+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9340ce6b0 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.598+0000 7fc93ffff700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9280a1590 0x7fc9280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.599+0000 7fc93d7fa700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] conn(0x7fc9280ab150 0x7fc9280ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.599+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 --> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc9280adcc0 con 0x7fc9280ab150 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.599+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9340cea20 con 0x7fc940075550 2023-12-11T19:32:25.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.599+0000 7fc944f3e700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] conn(0x7fc9280ab150 0x7fc9280ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:25.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.599+0000 7fc93ffff700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9280a1590 0x7fc9280a3a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc930009c60 tx=0x7fc930009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:25.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.600+0000 7fc944f3e700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] conn(0x7fc9280ab150 0x7fc9280ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:25.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.601+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== osd.1 v2:172.21.15.70:6800/1682554189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc9280adcc0 con 0x7fc9280ab150 2023-12-11T19:32:25.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.620+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 --> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc940111930 con 0x7fc9280ab150 2023-12-11T19:32:25.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.620+0000 7fc93d7fa700 1 -- 172.21.15.66:0/1677385443 <== osd.1 v2:172.21.15.70:6800/1682554189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc940111930 con 0x7fc9280ab150 2023-12-11T19:32:25.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] conn(0x7fc9280ab150 msgr2=0x7fc9280ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] conn(0x7fc9280ab150 0x7fc9280ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9280a1590 msgr2=0x7fc9280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9280a1590 0x7fc9280a3a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc930009c60 tx=0x7fc930009260 comp rx=0 tx=0).stop 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940075550 msgr2=0x7fc94019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 --2- 172.21.15.66:0/1677385443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc940075550 0x7fc94019e4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc934004760 tx=0x7fc934004790 comp rx=0 tx=0).stop 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc944f3e700 1 -- 172.21.15.66:0/1677385443 reap_dead start 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 shutdown_connections 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 >> 172.21.15.66:0/1677385443 conn(0x7fc9400787c0 msgr2=0x7fc940110fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 shutdown_connections 2023-12-11T19:32:25.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.621+0000 7fc9469a1700 1 -- 172.21.15.66:0/1677385443 wait complete. 2023-12-11T19:32:25.825 INFO:teuthology.orchestra.run.smithi066.stdout:111669149759 2023-12-11T19:32:25.825 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-11T19:32:25.863 INFO:teuthology.orchestra.run.smithi066.stdout:257698037796 2023-12-11T19:32:25.864 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-11T19:32:26.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:25 smithi167 ceph-mon[102750]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s; 191 B/s, 0 keys/s, 0 objects/s recovering 2023-12-11T19:32:26.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:25 smithi070 ceph-mon[104044]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s; 191 B/s, 0 keys/s, 0 objects/s recovering 2023-12-11T19:32:26.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 -- 172.21.15.66:0/1036767912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 msgr2=0x7f5b3010e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 --2- 172.21.15.66:0/1036767912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b3010e420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b0d0 tx=0x7f5b2800b3e0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 -- 172.21.15.66:0/1036767912 shutdown_connections 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 --2- 172.21.15.66:0/1036767912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b30107620 0x7f5b30107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 --2- 172.21.15.66:0/1036767912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b3010e420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 --2- 172.21.15.66:0/1036767912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b30068f70 0x7f5b300693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 -- 172.21.15.66:0/1036767912 >> 172.21.15.66:0/1036767912 conn(0x7f5b30076570 msgr2=0x7f5b30076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.644+0000 7f5b34edf700 1 -- 172.21.15.66:0/1036767912 shutdown_connections 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.645+0000 7f5b34edf700 1 -- 172.21.15.66:0/1036767912 wait complete. 2023-12-11T19:32:26.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.646+0000 7f5b34edf700 1 Processor -- start 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.646+0000 7f5b34edf700 1 -- start start 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b34edf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b30068f70 0x7f5b30100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b34edf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b30100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b34edf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b30107620 0x7f5b30101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b34edf700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301a5dc0 con 0x7f5b30069930 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b34edf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301a5f30 con 0x7f5b30068f70 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b34edf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301a60f0 con 0x7f5b30107620 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b30100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b30100b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60770/0 (socket says 172.21.15.66:60770) 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 -- 172.21.15.66:0/3601375643 learned_addr learned my addr 172.21.15.66:0/3601375643 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2e59c700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b30068f70 0x7f5b30100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2ed9d700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b30107620 0x7f5b30101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 -- 172.21.15.66:0/3601375643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b30107620 msgr2=0x7f5b30101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:26.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b30107620 0x7f5b30101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 -- 172.21.15.66:0/3601375643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b30068f70 msgr2=0x7f5b30100600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b30068f70 0x7f5b30100600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2dd9b700 1 -- 172.21.15.66:0/3601375643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009d00 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.647+0000 7f5b2e59c700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b30068f70 0x7f5b30100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.648+0000 7f5b2dd9b700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b30100b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400d950 tx=0x7f5b2400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.648+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24009b40 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.648+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b24009ca0 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.648+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b301a63d0 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.648+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400f620 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.648+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b301a68d0 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.649+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b30111670 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.651+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5b24010460 con 0x7f5b30069930 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.652+0000 7f5b1f7fe700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5b180a1510 0x7f5b180a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:26.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5b240cd380 con 0x7f5b30069930 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b2e59c700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5b180a1510 0x7f5b180a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b1f7fe700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] conn(0x7f5b180ab0d0 0x7f5b180ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 --> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b180adc40 con 0x7f5b180ab0d0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b240cd6f0 con 0x7f5b30069930 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b2ed9d700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] conn(0x7f5b180ab0d0 0x7f5b180ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.653+0000 7f5b2e59c700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5b180a1510 0x7f5b180a39d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005e10 tx=0x7f5b20005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.654+0000 7f5b2ed9d700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] conn(0x7f5b180ab0d0 0x7f5b180ad570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.655+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== osd.9 v2:172.21.15.66:6826/969904470 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5b180adc40 con 0x7f5b180ab0d0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.672+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 --> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b30061cd0 con 0x7f5b180ab0d0 2023-12-11T19:32:26.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b1f7fe700 1 -- 172.21.15.66:0/3601375643 <== osd.9 v2:172.21.15.66:6826/969904470 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5b30061cd0 con 0x7f5b180ab0d0 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] conn(0x7f5b180ab0d0 msgr2=0x7f5b180ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] conn(0x7f5b180ab0d0 0x7f5b180ad570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5b180a1510 msgr2=0x7f5b180a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5b180a1510 0x7f5b180a39d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005e10 tx=0x7f5b20005d60 comp rx=0 tx=0).stop 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 msgr2=0x7f5b30100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 --2- 172.21.15.66:0/3601375643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b30069930 0x7f5b30100b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400d950 tx=0x7f5b2400dd10 comp rx=0 tx=0).stop 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b2ed9d700 1 -- 172.21.15.66:0/3601375643 reap_dead start 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 shutdown_connections 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 >> 172.21.15.66:0/3601375643 conn(0x7f5b30076570 msgr2=0x7f5b300ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 shutdown_connections 2023-12-11T19:32:26.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:25.673+0000 7f5b34edf700 1 -- 172.21.15.66:0/3601375643 wait complete. 2023-12-11T19:32:27.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/2876798448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f41076f0 msgr2=0x7fe5f4107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/2876798448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f41076f0 0x7fe5f4107ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009990 tx=0x7fe5f0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/2876798448 shutdown_connections 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/2876798448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f4102040 0x7fe5f410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/2876798448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f41076f0 0x7fe5f4107ad0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/2876798448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f4101680 0x7fe5f4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/2876798448 >> 172.21.15.66:0/2876798448 conn(0x7fe5f40787c0 msgr2=0x7fe5f4078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/2876798448 shutdown_connections 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.058+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/2876798448 wait complete. 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.059+0000 7fe5fc5f0700 1 Processor -- start 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.059+0000 7fe5fc5f0700 1 -- start start 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.059+0000 7fe5fc5f0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f4101680 0x7fe5f419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.059+0000 7fe5fc5f0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f4102040 0x7fe5f419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fc5f0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f41076f0 0x7fe5f4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fc5f0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4197e00 con 0x7fe5f4101680 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fc5f0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4197f70 con 0x7fe5f41076f0 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fc5f0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4198130 con 0x7fe5f4102040 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f4101680 0x7fe5f419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f4101680 0x7fe5f419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60790/0 (socket says 172.21.15.66:60790) 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 -- 172.21.15.66:0/1985124390 learned_addr learned my addr 172.21.15.66:0/1985124390 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fab8d700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f41076f0 0x7fe5f4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 -- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f4102040 msgr2=0x7fe5f419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f4102040 0x7fe5f419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 -- 172.21.15.66:0/1985124390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f41076f0 msgr2=0x7fe5f4197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f41076f0 0x7fe5f4197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 -- 172.21.15.66:0/1985124390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00095f0 con 0x7fe5f4101680 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.060+0000 7fe5fa38c700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f4101680 0x7fe5f419d650 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00036c0 tx=0x7fe5f0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.061+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f001d070 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.061+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5f000fc00 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.061+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0022c40 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.061+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f41983b0 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.061+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f41a4640 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.062+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5f4198630 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.064+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe5f0022470 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.065+0000 7fe5eb7fe700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5e00a1590 0x7fe5e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.065+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5f00cea40 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.065+0000 7fe5f9b8b700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5e00a1590 0x7fe5e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.066+0000 7fe5eb7fe700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] conn(0x7fe5e00ab150 0x7fe5e00ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.066+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 --> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5e00adcc0 con 0x7fe5e00ab150 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.066+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5f00cedb0 con 0x7fe5f4101680 2023-12-11T19:32:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.066+0000 7fe5f9b8b700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5e00a1590 0x7fe5e00a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4005fd0 tx=0x7fe5e4005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.066+0000 7fe5fab8d700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] conn(0x7fe5e00ab150 0x7fe5e00ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.067+0000 7fe5fab8d700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] conn(0x7fe5e00ab150 0x7fe5e00ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.068+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== osd.2 v2:172.21.15.167:6800/440232508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe5e00adcc0 con 0x7fe5e00ab150 2023-12-11T19:32:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.086+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 --> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe5f4061c50 con 0x7fe5e00ab150 2023-12-11T19:32:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5eb7fe700 1 -- 172.21.15.66:0/1985124390 <== osd.2 v2:172.21.15.167:6800/440232508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe5f4061c50 con 0x7fe5e00ab150 2023-12-11T19:32:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] conn(0x7fe5e00ab150 msgr2=0x7fe5e00ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] conn(0x7fe5e00ab150 0x7fe5e00ad5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5e00a1590 msgr2=0x7fe5e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fe5e00a1590 0x7fe5e00a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4005fd0 tx=0x7fe5e4005d00 comp rx=0 tx=0).stop 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f4101680 msgr2=0x7fe5f419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 --2- 172.21.15.66:0/1985124390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5f4101680 0x7fe5f419d650 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00036c0 tx=0x7fe5f0005e80 comp rx=0 tx=0).stop 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fab8d700 1 -- 172.21.15.66:0/1985124390 reap_dead start 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 shutdown_connections 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 >> 172.21.15.66:0/1985124390 conn(0x7fe5f40787c0 msgr2=0x7fe5f4100340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 shutdown_connections 2023-12-11T19:32:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.087+0000 7fe5fc5f0700 1 -- 172.21.15.66:0/1985124390 wait complete. 2023-12-11T19:32:27.561 INFO:teuthology.orchestra.run.smithi066.stdout:292057776159 2023-12-11T19:32:27.562 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 -- 172.21.15.66:0/3689579409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8069b80 msgr2=0x7f35d81061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 --2- 172.21.15.66:0/3689579409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8069b80 0x7f35d81061b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009990 tx=0x7f35c4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 -- 172.21.15.66:0/3689579409 shutdown_connections 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 --2- 172.21.15.66:0/3689579409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35d8106890 0x7f35d810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 --2- 172.21.15.66:0/3689579409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 0x7f35d80695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 --2- 172.21.15.66:0/3689579409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8069b80 0x7f35d81061b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.752+0000 7f35defd9700 1 -- 172.21.15.66:0/3689579409 >> 172.21.15.66:0/3689579409 conn(0x7f35d8076570 msgr2=0x7f35d8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.753+0000 7f35defd9700 1 -- 172.21.15.66:0/3689579409 shutdown_connections 2023-12-11T19:32:27.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.753+0000 7f35defd9700 1 -- 172.21.15.66:0/3689579409 wait complete. 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.754+0000 7f35defd9700 1 Processor -- start 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.754+0000 7f35defd9700 1 -- start start 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35defd9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 0x7f35d819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35defd9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35d8069b80 0x7f35d819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35defd9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8106890 0x7f35d8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35defd9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d819a070 con 0x7f35d80691d0 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35defd9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d819a1e0 con 0x7f35d8106890 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35defd9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d819a3a0 con 0x7f35d8069b80 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35dcd75700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 0x7f35d819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35dcd75700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 0x7f35d819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60810/0 (socket says 172.21.15.66:60810) 2023-12-11T19:32:27.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.755+0000 7f35dcd75700 1 -- 172.21.15.66:0/3892030701 learned_addr learned my addr 172.21.15.66:0/3892030701 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dd576700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8106890 0x7f35d8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dcd75700 1 -- 172.21.15.66:0/3892030701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35d8069b80 msgr2=0x7f35d819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dcd75700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35d8069b80 0x7f35d819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dcd75700 1 -- 172.21.15.66:0/3892030701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8106890 msgr2=0x7f35d8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dcd75700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d8106890 0x7f35d8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dcd75700 1 -- 172.21.15.66:0/3892030701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c40095f0 con 0x7f35d80691d0 2023-12-11T19:32:27.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.756+0000 7f35dcd75700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 0x7f35d819f8d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00d950 tx=0x7f35cc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:27.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.757+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc009b40 con 0x7f35d80691d0 2023-12-11T19:32:27.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.757+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35cc009ca0 con 0x7f35d80691d0 2023-12-11T19:32:27.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.757+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00f620 con 0x7f35d80691d0 2023-12-11T19:32:27.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.757+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d819a680 con 0x7f35d80691d0 2023-12-11T19:32:27.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.757+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d81a68c0 con 0x7f35d80691d0 2023-12-11T19:32:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.758+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35d8111690 con 0x7f35d80691d0 2023-12-11T19:32:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.761+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f35cc010460 con 0x7f35d80691d0 2023-12-11T19:32:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.762+0000 7f35d5ffb700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35c80a1590 0x7f35c80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.762+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35cc0cd1d0 con 0x7f35d80691d0 2023-12-11T19:32:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.763+0000 7f35d5ffb700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] conn(0x7f35c80ab150 0x7f35c80ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.763+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 --> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35c80adcc0 con 0x7f35c80ab150 2023-12-11T19:32:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.763+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35cc0cf830 con 0x7f35d80691d0 2023-12-11T19:32:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.763+0000 7f35d7fff700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35c80a1590 0x7f35c80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.763+0000 7f35dd576700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] conn(0x7f35c80ab150 0x7f35c80ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.764+0000 7f35d7fff700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35c80a1590 0x7f35c80a3a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35c40098e0 tx=0x7f35c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.764+0000 7f35dd576700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] conn(0x7f35c80ab150 0x7f35c80ad5f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.765+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== osd.6 v2:172.21.15.66:6818/7519239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f35c80adcc0 con 0x7f35c80ab150 2023-12-11T19:32:27.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.785+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 --> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35d810d230 con 0x7f35c80ab150 2023-12-11T19:32:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.785+0000 7f35d5ffb700 1 -- 172.21.15.66:0/3892030701 <== osd.6 v2:172.21.15.66:6818/7519239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f35d810d230 con 0x7f35c80ab150 2023-12-11T19:32:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] conn(0x7f35c80ab150 msgr2=0x7f35c80ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35defd9700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] conn(0x7f35c80ab150 0x7f35c80ad5f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35c80a1590 msgr2=0x7f35c80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35defd9700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f35c80a1590 0x7f35c80a3a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35c40098e0 tx=0x7f35c4005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 msgr2=0x7f35d819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35defd9700 1 --2- 172.21.15.66:0/3892030701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d80691d0 0x7f35d819f8d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00d950 tx=0x7f35cc00dd10 comp rx=0 tx=0).stop 2023-12-11T19:32:27.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.786+0000 7f35dd576700 1 -- 172.21.15.66:0/3892030701 reap_dead start 2023-12-11T19:32:27.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.787+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 shutdown_connections 2023-12-11T19:32:27.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.787+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 >> 172.21.15.66:0/3892030701 conn(0x7f35d8076570 msgr2=0x7f35d810c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:27.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.787+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 shutdown_connections 2023-12-11T19:32:27.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:27.787+0000 7f35defd9700 1 -- 172.21.15.66:0/3892030701 wait complete. 2023-12-11T19:32:27.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:27 smithi066 ceph-mon[97920]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 10 op/s; 97 B/s, 0 objects/s recovering 2023-12-11T19:32:27.869 INFO:teuthology.orchestra.run.smithi066.stdout:55834574921 2023-12-11T19:32:27.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-11T19:32:28.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:27 smithi167 ceph-mon[102750]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 10 op/s; 97 B/s, 0 objects/s recovering 2023-12-11T19:32:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:27 smithi070 ceph-mon[104044]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 10 op/s; 97 B/s, 0 objects/s recovering 2023-12-11T19:32:28.443 INFO:teuthology.orchestra.run.smithi066.stdout:77309411398 2023-12-11T19:32:28.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-11T19:32:28.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.724+0000 7f48cff32700 1 -- 172.21.15.66:0/1653561062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8107620 msgr2=0x7f48c8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.724+0000 7f48cff32700 1 --2- 172.21.15.66:0/1653561062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8107620 0x7f48c8107a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f48c40098f0 tx=0x7f48c4009c00 comp rx=0 tx=0).stop 2023-12-11T19:32:28.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 -- 172.21.15.66:0/1653561062 shutdown_connections 2023-12-11T19:32:28.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 --2- 172.21.15.66:0/1653561062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8069930 0x7f48c810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 --2- 172.21.15.66:0/1653561062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8107620 0x7f48c8107a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 --2- 172.21.15.66:0/1653561062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c8068f70 0x7f48c80693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 -- 172.21.15.66:0/1653561062 >> 172.21.15.66:0/1653561062 conn(0x7f48c8076570 msgr2=0x7f48c8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 -- 172.21.15.66:0/1653561062 shutdown_connections 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.725+0000 7f48cff32700 1 -- 172.21.15.66:0/1653561062 wait complete. 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.726+0000 7f48cff32700 1 Processor -- start 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 -- start start 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8068f70 0x7f48c819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8069930 0x7f48c819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c8107620 0x7f48c81a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81a2e40 con 0x7f48c8107620 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81a2fb0 con 0x7f48c8068f70 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cff32700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81a3170 con 0x7f48c8069930 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.727+0000 7f48cdcce700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8068f70 0x7f48c819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8068f70 0x7f48c819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36364/0 (socket says 172.21.15.66:36364) 2023-12-11T19:32:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 -- 172.21.15.66:0/3720373825 learned_addr learned my addr 172.21.15.66:0/3720373825 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48ce4cf700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c8107620 0x7f48c81a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cd4cd700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8069930 0x7f48c819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 -- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8069930 msgr2=0x7f48c819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c8069930 0x7f48c819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 -- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c8107620 msgr2=0x7f48c81a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48c8107620 0x7f48c81a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 -- 172.21.15.66:0/3720373825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b800f750 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.728+0000 7f48cdcce700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8068f70 0x7f48c819df40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009230 tx=0x7f48c400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.729+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400b8f0 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.729+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48c400ba50 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.729+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4003c40 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.730+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c4009520 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.730+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c81a3680 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.731+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48c810f4a0 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.734+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f48c4005040 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.736+0000 7f48beffd700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48b40a1610 0x7f48b40a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.736+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f48c40ce450 con 0x7f48c8068f70 2023-12-11T19:32:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.736+0000 7f48cd4cd700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48b40a1610 0x7f48b40a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.737+0000 7f48beffd700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] conn(0x7f48b40ab1d0 0x7f48b40ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.737+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 --> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48b40add40 con 0x7f48b40ab1d0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.737+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48c40ce800 con 0x7f48c8068f70 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.737+0000 7f48cd4cd700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48b40a1610 0x7f48b40a3ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48c819f4a0 tx=0x7f48b8011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.737+0000 7f48ce4cf700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] conn(0x7f48b40ab1d0 0x7f48b40ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.738+0000 7f48ce4cf700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] conn(0x7f48b40ab1d0 0x7f48b40ad670 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.739+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== osd.11 v2:172.21.15.167:6824/819959610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f48b40add40 con 0x7f48b40ab1d0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.757+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 --> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48c8102cc0 con 0x7f48b40ab1d0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.759+0000 7f48beffd700 1 -- 172.21.15.66:0/3720373825 <== osd.11 v2:172.21.15.167:6824/819959610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f48c8102cc0 con 0x7f48b40ab1d0 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.759+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] conn(0x7f48b40ab1d0 msgr2=0x7f48b40ad670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.759+0000 7f48cff32700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] conn(0x7f48b40ab1d0 0x7f48b40ad670 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48b40a1610 msgr2=0x7f48b40a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f48b40a1610 0x7f48b40a3ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48c819f4a0 tx=0x7f48b8011040 comp rx=0 tx=0).stop 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8068f70 msgr2=0x7f48c819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 --2- 172.21.15.66:0/3720373825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c8068f70 0x7f48c819df40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009230 tx=0x7f48c400fa00 comp rx=0 tx=0).stop 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48ce4cf700 1 -- 172.21.15.66:0/3720373825 reap_dead start 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 shutdown_connections 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 >> 172.21.15.66:0/3720373825 conn(0x7f48c8076570 msgr2=0x7f48c80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 shutdown_connections 2023-12-11T19:32:28.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.760+0000 7f48cff32700 1 -- 172.21.15.66:0/3720373825 wait complete. 2023-12-11T19:32:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:28.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.868+0000 7f68ed37b700 1 -- 172.21.15.66:0/3394279529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8068f70 msgr2=0x7f68e80693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.868+0000 7f68ed37b700 1 --2- 172.21.15.66:0/3394279529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8068f70 0x7f68e80693f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f68d00099c0 tx=0x7f68d0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 -- 172.21.15.66:0/3394279529 shutdown_connections 2023-12-11T19:32:28.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 --2- 172.21.15.66:0/3394279529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68e8069930 0x7f68e810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 --2- 172.21.15.66:0/3394279529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 0x7f68e8107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 --2- 172.21.15.66:0/3394279529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8068f70 0x7f68e80693f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 -- 172.21.15.66:0/3394279529 >> 172.21.15.66:0/3394279529 conn(0x7f68e8076570 msgr2=0x7f68e8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:28.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 -- 172.21.15.66:0/3394279529 shutdown_connections 2023-12-11T19:32:28.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.869+0000 7f68ed37b700 1 -- 172.21.15.66:0/3394279529 wait complete. 2023-12-11T19:32:28.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.871+0000 7f68ed37b700 1 Processor -- start 2023-12-11T19:32:28.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.871+0000 7f68ed37b700 1 -- start start 2023-12-11T19:32:28.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.872+0000 7f68ed37b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68e8068f70 0x7f68e8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.872+0000 7f68ed37b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8069930 0x7f68e819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.872+0000 7f68ed37b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 0x7f68e819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.872+0000 7f68ed37b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819eb10 con 0x7f68e8068f70 2023-12-11T19:32:28.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.872+0000 7f68ed37b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819ec50 con 0x7f68e8069930 2023-12-11T19:32:28.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.872+0000 7f68ed37b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e819ee10 con 0x7f68e8107620 2023-12-11T19:32:28.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 0x7f68e819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 0x7f68e819e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:59686/0 (socket says 172.21.15.66:59686) 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 -- 172.21.15.66:0/1757483620 learned_addr learned my addr 172.21.15.66:0/1757483620 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68de7fc700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8069930 0x7f68e819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 -- 172.21.15.66:0/1757483620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8069930 msgr2=0x7f68e819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68e8069930 0x7f68e819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 -- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68e8068f70 msgr2=0x7f68e8199d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f68e8068f70 0x7f68e8199d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.873+0000 7f68e77fe700 1 -- 172.21.15.66:0/1757483620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0009520 con 0x7f68e8107620 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.874+0000 7f68e77fe700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 0x7f68e819e4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68e000ea80 tx=0x7f68e000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.874+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e000cc10 con 0x7f68e8107620 2023-12-11T19:32:28.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.874+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d00095f0 con 0x7f68e8107620 2023-12-11T19:32:28.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.875+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f68e00044b0 con 0x7f68e8107620 2023-12-11T19:32:28.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.875+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0003d90 con 0x7f68e8107620 2023-12-11T19:32:28.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.875+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8110a80 con 0x7f68e8107620 2023-12-11T19:32:28.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.878+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68e8101130 con 0x7f68e8107620 2023-12-11T19:32:28.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.879+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f68e0018430 con 0x7f68e8107620 2023-12-11T19:32:28.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.881+0000 7f68e4ff9700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f68d40a1570 0x7f68d40a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.882+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f68e0014070 con 0x7f68e8107620 2023-12-11T19:32:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.882+0000 7f68e4ff9700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] conn(0x7f68d40abd40 0x7f68d40ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.882+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 --> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68d40ae8b0 con 0x7f68d40abd40 2023-12-11T19:32:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.882+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68e0003d90 con 0x7f68e8107620 2023-12-11T19:32:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.883+0000 7f68de7fc700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] conn(0x7f68d40abd40 0x7f68d40ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.883+0000 7f68e6ffd700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f68d40a1570 0x7f68d40a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:28.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.884+0000 7f68e6ffd700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f68d40a1570 0x7f68d40a3a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f68d8007f20 tx=0x7f68d8007470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:28.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.885+0000 7f68de7fc700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] conn(0x7f68d40abd40 0x7f68d40ae1e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:28.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.887+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== osd.0 v2:172.21.15.66:6802/345409379 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f68d40ae8b0 con 0x7f68d40abd40 2023-12-11T19:32:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.915+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 --> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68e8102cc0 con 0x7f68d40abd40 2023-12-11T19:32:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.916+0000 7f68e4ff9700 1 -- 172.21.15.66:0/1757483620 <== osd.0 v2:172.21.15.66:6802/345409379 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f68e8102cc0 con 0x7f68d40abd40 2023-12-11T19:32:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.916+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] conn(0x7f68d40abd40 msgr2=0x7f68d40ae1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.916+0000 7f68ed37b700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] conn(0x7f68d40abd40 0x7f68d40ae1e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f68d40a1570 msgr2=0x7f68d40a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f68d40a1570 0x7f68d40a3a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f68d8007f20 tx=0x7f68d8007470 comp rx=0 tx=0).stop 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 msgr2=0x7f68e819e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 --2- 172.21.15.66:0/1757483620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68e8107620 0x7f68e819e4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f68e000ea80 tx=0x7f68e000ee40 comp rx=0 tx=0).stop 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68e77fe700 1 -- 172.21.15.66:0/1757483620 reap_dead start 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 shutdown_connections 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 >> 172.21.15.66:0/1757483620 conn(0x7f68e8076570 msgr2=0x7f68e80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 shutdown_connections 2023-12-11T19:32:28.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:28.917+0000 7f68ed37b700 1 -- 172.21.15.66:0/1757483620 wait complete. 2023-12-11T19:32:29.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 -- 172.21.15.66:0/1113587698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 msgr2=0x7f3d50073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:29.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 --2- 172.21.15.66:0/1113587698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d50073800 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009990 tx=0x7f3d40009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 -- 172.21.15.66:0/1113587698 shutdown_connections 2023-12-11T19:32:29.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 --2- 172.21.15.66:0/1113587698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50112350 0x7f3d50114780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 --2- 172.21.15.66:0/1113587698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d50073800 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 --2- 172.21.15.66:0/1113587698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d50073d40 0x7f3d500741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.096+0000 7f3d56677700 1 -- 172.21.15.66:0/1113587698 >> 172.21.15.66:0/1113587698 conn(0x7f3d500787c0 msgr2=0x7f3d50078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:29.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.097+0000 7f3d56677700 1 -- 172.21.15.66:0/1113587698 shutdown_connections 2023-12-11T19:32:29.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.097+0000 7f3d56677700 1 -- 172.21.15.66:0/1113587698 wait complete. 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.098+0000 7f3d56677700 1 Processor -- start 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.098+0000 7f3d56677700 1 -- start start 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.098+0000 7f3d56677700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d50073d40 0x7f3d5019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d56677700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d5019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d56677700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50112350 0x7f3d501a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d56677700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501a2f60 con 0x7f3d50075550 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d56677700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501a30d0 con 0x7f3d50073d40 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d56677700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d501a3290 con 0x7f3d50112350 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d4f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d5019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:29.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d4f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d5019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60850/0 (socket says 172.21.15.66:60850) 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d4f7fe700 1 -- 172.21.15.66:0/2207735199 learned_addr learned my addr 172.21.15.66:0/2207735199 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d54c14700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50112350 0x7f3d501a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.099+0000 7f3d4ffff700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d50073d40 0x7f3d5019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4f7fe700 1 -- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50112350 msgr2=0x7f3d501a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4f7fe700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d50112350 0x7f3d501a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4f7fe700 1 -- 172.21.15.66:0/2207735199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d50073d40 msgr2=0x7f3d5019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4f7fe700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d50073d40 0x7f3d5019e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4f7fe700 1 -- 172.21.15.66:0/2207735199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d400095f0 con 0x7f3d50075550 2023-12-11T19:32:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4f7fe700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d5019e5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400d950 tx=0x7f3d4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400f890 con 0x7f3d50075550 2023-12-11T19:32:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d501a3570 con 0x7f3d50075550 2023-12-11T19:32:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d4400fe80 con 0x7f3d50075550 2023-12-11T19:32:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400e3b0 con 0x7f3d50075550 2023-12-11T19:32:29.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.100+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d501aadd0 con 0x7f3d50075550 2023-12-11T19:32:29.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.101+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d50117d90 con 0x7f3d50075550 2023-12-11T19:32:29.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.104+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3d44010460 con 0x7f3d50075550 2023-12-11T19:32:29.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.106+0000 7f3d4d7fa700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d380a1590 0x7f3d380a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:29.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.106+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3d440ccf70 con 0x7f3d50075550 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.106+0000 7f3d4ffff700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d380a1590 0x7f3d380a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.106+0000 7f3d4d7fa700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] conn(0x7f3d380ab150 0x7f3d380ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.106+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 --> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d380adcc0 con 0x7f3d380ab150 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.106+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d440cd360 con 0x7f3d50075550 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.107+0000 7f3d4ffff700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d380a1590 0x7f3d380a3a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000b5c0 tx=0x7f3d40005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.107+0000 7f3d54c14700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] conn(0x7f3d380ab150 0x7f3d380ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:29.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.108+0000 7f3d54c14700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] conn(0x7f3d380ab150 0x7f3d380ad5f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:29.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.109+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== osd.5 v2:172.21.15.167:6808/1023707142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3d380adcc0 con 0x7f3d380ab150 2023-12-11T19:32:29.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.127+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 --> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d501119d0 con 0x7f3d380ab150 2023-12-11T19:32:29.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.128+0000 7f3d4d7fa700 1 -- 172.21.15.66:0/2207735199 <== osd.5 v2:172.21.15.167:6808/1023707142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d501119d0 con 0x7f3d380ab150 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.128+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] conn(0x7f3d380ab150 msgr2=0x7f3d380ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.128+0000 7f3d56677700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] conn(0x7f3d380ab150 0x7f3d380ad5f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d380a1590 msgr2=0x7f3d380a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3d380a1590 0x7f3d380a3a50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000b5c0 tx=0x7f3d40005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 msgr2=0x7f3d5019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 --2- 172.21.15.66:0/2207735199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d50075550 0x7f3d5019e5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400d950 tx=0x7f3d4400dc60 comp rx=0 tx=0).stop 2023-12-11T19:32:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d54c14700 1 -- 172.21.15.66:0/2207735199 reap_dead start 2023-12-11T19:32:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 shutdown_connections 2023-12-11T19:32:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 >> 172.21.15.66:0/2207735199 conn(0x7f3d500787c0 msgr2=0x7f3d50111080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 shutdown_connections 2023-12-11T19:32:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:29.129+0000 7f3d56677700 1 -- 172.21.15.66:0/2207735199 wait complete. 2023-12-11T19:32:29.541 INFO:teuthology.orchestra.run.smithi066.stdout:201863462961 2023-12-11T19:32:29.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-11T19:32:29.630 INFO:teuthology.orchestra.run.smithi066.stdout:347892350997 2023-12-11T19:32:29.631 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-11T19:32:29.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:29 smithi066 ceph-mon[97920]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-12-11T19:32:29.991 INFO:teuthology.orchestra.run.smithi066.stdout:34359738450 2023-12-11T19:32:29.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-11T19:32:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:29 smithi167 ceph-mon[102750]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-12-11T19:32:30.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:29 smithi070 ceph-mon[104044]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2023-12-11T19:32:30.312 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:30.351 INFO:teuthology.orchestra.run.smithi066.stdout:167503724598 2023-12-11T19:32:30.352 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-11T19:32:30.363 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:30.515 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:30.522 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:30.522 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:31.686 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:31 smithi066 ceph-mon[97920]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-11T19:32:31.770 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:32.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:31 smithi167 ceph-mon[102750]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-11T19:32:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:31 smithi070 ceph-mon[104044]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-11T19:32:32.225 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 -- 172.21.15.66:0/2964503321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec701065a0 msgr2=0x7fec70106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 --2- 172.21.15.66:0/2964503321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec701065a0 0x7fec70106980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec60009990 tx=0x7fec60009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 -- 172.21.15.66:0/2964503321 shutdown_connections 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 --2- 172.21.15.66:0/2964503321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec70101fe0 0x7fec7010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 --2- 172.21.15.66:0/2964503321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec701065a0 0x7fec70106980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 --2- 172.21.15.66:0/2964503321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec70101620 0x7fec70101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 -- 172.21.15.66:0/2964503321 >> 172.21.15.66:0/2964503321 conn(0x7fec70078730 msgr2=0x7fec70078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 -- 172.21.15.66:0/2964503321 shutdown_connections 2023-12-11T19:32:33.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.126+0000 7fec74dd6700 1 -- 172.21.15.66:0/2964503321 wait complete. 2023-12-11T19:32:33.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.127+0000 7fec74dd6700 1 Processor -- start 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 -- start start 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec70101620 0x7fec700738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 0x7fec70073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec701065a0 0x7fec70074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701a5c10 con 0x7fec70101fe0 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701a5d80 con 0x7fec70101620 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec74dd6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701a5f40 con 0x7fec701065a0 2023-12-11T19:32:33.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec6dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 0x7fec70073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:33.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec6dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 0x7fec70073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56664/0 (socket says 172.21.15.66:56664) 2023-12-11T19:32:33.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.128+0000 7fec6dd9b700 1 -- 172.21.15.66:0/1553089716 learned_addr learned my addr 172.21.15.66:0/1553089716 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:33.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6e59c700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec70101620 0x7fec700738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6ed9d700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec701065a0 0x7fec70074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6dd9b700 1 -- 172.21.15.66:0/1553089716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec701065a0 msgr2=0x7fec70074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6dd9b700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec701065a0 0x7fec70074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6dd9b700 1 -- 172.21.15.66:0/1553089716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec70101620 msgr2=0x7fec700738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6dd9b700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec70101620 0x7fec700738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6dd9b700 1 -- 172.21.15.66:0/1553089716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600095f0 con 0x7fec70101fe0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec6dd9b700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 0x7fec70073e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c9b0 tx=0x7fec6400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64007b00 con 0x7fec70101fe0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec701a6220 con 0x7fec70101fe0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.129+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec6400ce50 con 0x7fec70101fe0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.130+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64018400 con 0x7fec70101fe0 2023-12-11T19:32:33.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.130+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701a6770 con 0x7fec70101fe0 2023-12-11T19:32:33.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.131+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec70109b90 con 0x7fec70101fe0 2023-12-11T19:32:33.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.135+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fec6400f450 con 0x7fec70101fe0 2023-12-11T19:32:33.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.136+0000 7fec5f7fe700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec580a1590 0x7fec580a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:33.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.136+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fec640ccd20 con 0x7fec70101fe0 2023-12-11T19:32:33.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.137+0000 7fec6e59c700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec580a1590 0x7fec580a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:33.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.137+0000 7fec6e59c700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec580a1590 0x7fec580a3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b5c0 tx=0x7fec60005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:33.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.142+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fec64096220 con 0x7fec70101fe0 2023-12-11T19:32:33.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.387+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fec70102ca0 con 0x7fec70101fe0 2023-12-11T19:32:33.402 INFO:teuthology.orchestra.run.smithi066.stdout:223338299434 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.388+0000 7fec5f7fe700 1 -- 172.21.15.66:0/1553089716 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec6400f700 con 0x7fec70101fe0 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec580a1590 msgr2=0x7fec580a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec580a1590 0x7fec580a3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b5c0 tx=0x7fec60005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 msgr2=0x7fec70073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 0x7fec70073e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c9b0 tx=0x7fec6400ccc0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 shutdown_connections 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec580a1590 0x7fec580a3a50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec70101620 0x7fec700738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec701065a0 0x7fec70074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 --2- 172.21.15.66:0/1553089716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec70101fe0 0x7fec70073e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:33.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 >> 172.21.15.66:0/1553089716 conn(0x7fec70078730 msgr2=0x7fec700ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 shutdown_connections 2023-12-11T19:32:33.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:33.390+0000 7fec74dd6700 1 -- 172.21.15.66:0/1553089716 wait complete. 2023-12-11T19:32:33.884 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:33 smithi066 ceph-mon[97920]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:33.885 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:33 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1553089716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-11T19:32:34.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:33 smithi167 ceph-mon[102750]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:34.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:33 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1553089716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-11T19:32:34.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:33 smithi070 ceph-mon[104044]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:34.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:33 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1553089716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-11T19:32:35.617 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299432 got 223338299434 for osd.7 2023-12-11T19:32:35.618 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 -- 172.21.15.66:0/1419395079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0105710 msgr2=0x7f53a0105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 --2- 172.21.15.66:0/1419395079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0105710 0x7f53a0105b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f539800b0a0 tx=0x7f539800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 -- 172.21.15.66:0/1419395079 shutdown_connections 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 --2- 172.21.15.66:0/1419395079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a0100110 0x7f53a01004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 --2- 172.21.15.66:0/1419395079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0105710 0x7f53a0105b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 --2- 172.21.15.66:0/1419395079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a0100a30 0x7f53a01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 -- 172.21.15.66:0/1419395079 >> 172.21.15.66:0/1419395079 conn(0x7f53a00fbe10 msgr2=0x7f53a00fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:35.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.668+0000 7f53a653b700 1 -- 172.21.15.66:0/1419395079 shutdown_connections 2023-12-11T19:32:35.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.669+0000 7f53a653b700 1 -- 172.21.15.66:0/1419395079 wait complete. 2023-12-11T19:32:35.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.670+0000 7f53a653b700 1 Processor -- start 2023-12-11T19:32:35.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.670+0000 7f53a653b700 1 -- start start 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.670+0000 7f53a653b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a0100110 0x7f53a019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a653b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 0x7f53a019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a653b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a0105710 0x7f53a01a2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a653b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a01a2e70 con 0x7f53a0100a30 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a653b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a01a2fe0 con 0x7f53a0100110 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a653b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a01a31a0 con 0x7f53a0105710 2023-12-11T19:32:35.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a4d38700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 0x7f53a019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:35.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a4d38700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 0x7f53a019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56692/0 (socket says 172.21.15.66:56692) 2023-12-11T19:32:35.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a4d38700 1 -- 172.21.15.66:0/3085839460 learned_addr learned my addr 172.21.15.66:0/3085839460 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:35.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a5d3a700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a0105710 0x7f53a01a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:35.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.671+0000 7f53a5539700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a0100110 0x7f53a019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:35.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a4d38700 1 -- 172.21.15.66:0/3085839460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a0105710 msgr2=0x7f53a01a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:35.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a4d38700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a0105710 0x7f53a01a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a4d38700 1 -- 172.21.15.66:0/3085839460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a0100110 msgr2=0x7f53a019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:35.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a4d38700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a0100110 0x7f53a019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a4d38700 1 -- 172.21.15.66:0/3085839460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398009d00 con 0x7f53a0100a30 2023-12-11T19:32:35.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a4d38700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 0x7f53a019e4b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9a0 tx=0x7f539000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:35.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390009790 con 0x7f53a0100a30 2023-12-11T19:32:35.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a01a3480 con 0x7f53a0100a30 2023-12-11T19:32:35.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5390010430 con 0x7f53a0100a30 2023-12-11T19:32:35.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539000cea0 con 0x7f53a0100a30 2023-12-11T19:32:35.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.672+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a01aace0 con 0x7f53a0100a30 2023-12-11T19:32:35.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.674+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a0069850 con 0x7f53a0100a30 2023-12-11T19:32:35.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.676+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f53900105a0 con 0x7f53a0100a30 2023-12-11T19:32:35.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.678+0000 7f53967fc700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f538c0a1590 0x7f538c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:35.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.678+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5390014070 con 0x7f53a0100a30 2023-12-11T19:32:35.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.678+0000 7f53a5539700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f538c0a1590 0x7f538c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:35.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.679+0000 7f53a5539700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f538c0a1590 0x7f538c0a3a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f539c009c60 tx=0x7f539c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:35.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.681+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5390096670 con 0x7f53a0100a30 2023-12-11T19:32:35.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.944+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53a004f290 con 0x7f53a0100a30 2023-12-11T19:32:35.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.944+0000 7f53967fc700 1 -- 172.21.15.66:0/3085839460 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5390026090 con 0x7f53a0100a30 2023-12-11T19:32:35.946 INFO:teuthology.orchestra.run.smithi066.stdout:55834574923 2023-12-11T19:32:35.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f538c0a1590 msgr2=0x7f538c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:35.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f538c0a1590 0x7f538c0a3a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f539c009c60 tx=0x7f539c009260 comp rx=0 tx=0).stop 2023-12-11T19:32:35.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 msgr2=0x7f53a019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:35.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 0x7f53a019e4b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9a0 tx=0x7f539000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 shutdown_connections 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f538c0a1590 0x7f538c0a3a50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a0100110 0x7f53a019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a0105710 0x7f53a01a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 --2- 172.21.15.66:0/3085839460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53a0100a30 0x7f53a019e4b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 >> 172.21.15.66:0/3085839460 conn(0x7f53a00fbe10 msgr2=0x7f53a01031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 shutdown_connections 2023-12-11T19:32:35.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:35.947+0000 7f53a653b700 1 -- 172.21.15.66:0/3085839460 wait complete. 2023-12-11T19:32:35.971 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:35 smithi066 ceph-mon[97920]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-11T19:32:36.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:35 smithi167 ceph-mon[102750]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-11T19:32:36.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:35 smithi070 ceph-mon[104044]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-11T19:32:36.287 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 -- 172.21.15.66:0/3092036522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 msgr2=0x7fee68107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/3092036522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68107ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee58009990 tx=0x7fee58009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 -- 172.21.15.66:0/3092036522 shutdown_connections 2023-12-11T19:32:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/3092036522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee68102040 0x7fee6810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/3092036522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68107ad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/3092036522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 0x7fee68101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 -- 172.21.15.66:0/3092036522 >> 172.21.15.66:0/3092036522 conn(0x7fee680787c0 msgr2=0x7fee68078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 -- 172.21.15.66:0/3092036522 shutdown_connections 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.188+0000 7fee6ccd3700 1 -- 172.21.15.66:0/3092036522 wait complete. 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.189+0000 7fee6ccd3700 1 Processor -- start 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.189+0000 7fee6ccd3700 1 -- start start 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6ccd3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 0x7fee6819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6ccd3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee68102040 0x7fee6819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6ccd3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6ccd3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68199ff0 con 0x7fee681076f0 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6ccd3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6819a160 con 0x7fee68102040 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6ccd3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6819a320 con 0x7fee68101680 2023-12-11T19:32:36.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 0x7fee6819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee66d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee6659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 0x7fee6819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:60336/0 (socket says 172.21.15.66:60336) 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee66d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56710/0 (socket says 172.21.15.66:56710) 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee66d9d700 1 -- 172.21.15.66:0/1908182514 learned_addr learned my addr 172.21.15.66:0/1908182514 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.190+0000 7fee65d9b700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee68102040 0x7fee6819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee66d9d700 1 -- 172.21.15.66:0/1908182514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 msgr2=0x7fee6819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee66d9d700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 0x7fee6819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee66d9d700 1 -- 172.21.15.66:0/1908182514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee68102040 msgr2=0x7fee6819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee66d9d700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee68102040 0x7fee6819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee66d9d700 1 -- 172.21.15.66:0/1908182514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580095f0 con 0x7fee681076f0 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee66d9d700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68199990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fee6000b9e0 tx=0x7fee6000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000d820 con 0x7fee681076f0 2023-12-11T19:32:36.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee6000de10 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6819a600 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60015e70 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.191+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee681a6840 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.192+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee68104940 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.195+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fee6000d980 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.196+0000 7fee577fe700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fee500a1590 0x7fee500a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.196+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fee60011070 con 0x7fee681076f0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.196+0000 7fee6659c700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fee500a1590 0x7fee500a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.197+0000 7fee6659c700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fee500a1590 0x7fee500a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b5c0 tx=0x7fee58005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.201+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fee600963f0 con 0x7fee681076f0 2023-12-11T19:32:36.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 -- 172.21.15.66:0/1913950264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 msgr2=0x7f99140693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 --2- 172.21.15.66:0/1913950264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f99140693f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f99040099c0 tx=0x7f9904009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 -- 172.21.15.66:0/1913950264 shutdown_connections 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 --2- 172.21.15.66:0/1913950264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 0x7f9914107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 --2- 172.21.15.66:0/1913950264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 --2- 172.21.15.66:0/1913950264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f99140693f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.383+0000 7f991b6bf700 1 -- 172.21.15.66:0/1913950264 >> 172.21.15.66:0/1913950264 conn(0x7f9914076570 msgr2=0x7f9914076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.384+0000 7f991b6bf700 1 -- 172.21.15.66:0/1913950264 shutdown_connections 2023-12-11T19:32:36.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.384+0000 7f991b6bf700 1 -- 172.21.15.66:0/1913950264 wait complete. 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.385+0000 7f991b6bf700 1 Processor -- start 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.385+0000 7f991b6bf700 1 -- start start 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991b6bf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f991419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991b6bf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991b6bf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 0x7f9914199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991b6bf700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991419a080 con 0x7f9914068f70 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991b6bf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991419a1f0 con 0x7f9914107620 2023-12-11T19:32:36.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991b6bf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991419a3b0 con 0x7f9914069930 2023-12-11T19:32:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991945b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f991419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f9918c5a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f9918c5a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:60348/0 (socket says 172.21.15.66:60348) 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f9918c5a700 1 -- 172.21.15.66:0/4222385142 learned_addr learned my addr 172.21.15.66:0/4222385142 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f991945b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f991419f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56732/0 (socket says 172.21.15.66:56732) 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.386+0000 7f9919c5c700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 0x7f9914199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9918c5a700 1 -- 172.21.15.66:0/4222385142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 msgr2=0x7f9914199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9918c5a700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 0x7f9914199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9918c5a700 1 -- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 msgr2=0x7f991419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9918c5a700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f991419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9918c5a700 1 -- 172.21.15.66:0/4222385142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040095f0 con 0x7f9914069930 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9919c5c700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 0x7f9914199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:36.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f9918c5a700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991419fe20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9904005910 tx=0x7f9904005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990401d070 con 0x7f9914069930 2023-12-11T19:32:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f991b6bf700 1 -- 172.21.15.66:0/4222385142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991419a630 con 0x7f9914069930 2023-12-11T19:32:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9904004ba0 con 0x7f9914069930 2023-12-11T19:32:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.387+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400fa10 con 0x7f9914069930 2023-12-11T19:32:36.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.388+0000 7f991b6bf700 1 -- 172.21.15.66:0/4222385142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99141a68d0 con 0x7f9914069930 2023-12-11T19:32:36.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.389+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f80052f0 con 0x7f9914069930 2023-12-11T19:32:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.391+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f990400fb90 con 0x7f9914069930 2023-12-11T19:32:36.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.392+0000 7f990a7fc700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99000a1590 0x7f99000a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:36.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.393+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99040cdad0 con 0x7f9914069930 2023-12-11T19:32:36.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.393+0000 7f991945b700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99000a1590 0x7f99000a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:36.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.393+0000 7f991945b700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99000a1590 0x7f99000a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9910009be0 tx=0x7f9910009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:36.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.397+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9904097020 con 0x7f9914069930 2023-12-11T19:32:36.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.466+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fee68102d10 con 0x7fee681076f0 2023-12-11T19:32:36.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.466+0000 7fee577fe700 1 -- 172.21.15.66:0/1908182514 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee60023090 con 0x7fee681076f0 2023-12-11T19:32:36.472 INFO:teuthology.orchestra.run.smithi066.stdout:257698037798 2023-12-11T19:32:36.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fee500a1590 msgr2=0x7fee500a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fee500a1590 0x7fee500a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b5c0 tx=0x7fee58005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 msgr2=0x7fee68199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68199990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fee6000b9e0 tx=0x7fee6000bcf0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 shutdown_connections 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fee500a1590 0x7fee500a3a50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee681076f0 0x7fee68199990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68101680 0x7fee6819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 --2- 172.21.15.66:0/1908182514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee68102040 0x7fee6819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 >> 172.21.15.66:0/1908182514 conn(0x7fee680787c0 msgr2=0x7fee6810caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:36.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 shutdown_connections 2023-12-11T19:32:36.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.469+0000 7fee6ccd3700 1 -- 172.21.15.66:0/1908182514 wait complete. 2023-12-11T19:32:36.622 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:36.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.652+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f98f8005160 con 0x7f9914069930 2023-12-11T19:32:36.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.652+0000 7f990a7fc700 1 -- 172.21.15.66:0/4222385142 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f990409a640 con 0x7f9914069930 2023-12-11T19:32:36.654 INFO:teuthology.orchestra.run.smithi066.stdout:111669149762 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99000a1590 msgr2=0x7f99000a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99000a1590 0x7f99000a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9910009be0 tx=0x7f9910009260 comp rx=0 tx=0).stop 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 msgr2=0x7f991419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991419fe20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9904005910 tx=0x7f9904005270 comp rx=0 tx=0).stop 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 shutdown_connections 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99000a1590 0x7f99000a3a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9914107620 0x7f9914199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9914068f70 0x7f991419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 --2- 172.21.15.66:0/4222385142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9914069930 0x7f991419fe20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:36.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.655+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 >> 172.21.15.66:0/4222385142 conn(0x7f9914076570 msgr2=0x7f99140ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:36.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.656+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 shutdown_connections 2023-12-11T19:32:36.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:36.656+0000 7f98fffff700 1 -- 172.21.15.66:0/4222385142 wait complete. 2023-12-11T19:32:36.756 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:36 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3085839460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-11T19:32:36.756 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:36 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1908182514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-11T19:32:37.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:36 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3085839460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-11T19:32:37.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:36 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1908182514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-11T19:32:37.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:36 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3085839460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-11T19:32:37.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:36 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1908182514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-11T19:32:37.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 -- 172.21.15.66:0/1349190945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0568102040 msgr2=0x7f056810e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:37.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 --2- 172.21.15.66:0/1349190945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0568102040 0x7f056810e430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f056000b0a0 tx=0x7f056000b3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 -- 172.21.15.66:0/1349190945 shutdown_connections 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 --2- 172.21.15.66:0/1349190945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05681076f0 0x7f0568107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 --2- 172.21.15.66:0/1349190945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0568102040 0x7f056810e430 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 --2- 172.21.15.66:0/1349190945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0568101680 0x7f0568101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 -- 172.21.15.66:0/1349190945 >> 172.21.15.66:0/1349190945 conn(0x7f05680787c0 msgr2=0x7f0568078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 -- 172.21.15.66:0/1349190945 shutdown_connections 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.215+0000 7f056ff54700 1 -- 172.21.15.66:0/1349190945 wait complete. 2023-12-11T19:32:37.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.216+0000 7f056ff54700 1 Processor -- start 2023-12-11T19:32:37.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.216+0000 7f056ff54700 1 -- start start 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.216+0000 7f056ff54700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0568101680 0x7f05680738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.216+0000 7f056ff54700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0568102040 0x7f0568073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056ff54700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 0x7f0568074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056ff54700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681a5e60 con 0x7f05681076f0 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056ff54700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681a5fd0 con 0x7f0568101680 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056ff54700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05681a6190 con 0x7f0568102040 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 0x7f0568074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 0x7f0568074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56750/0 (socket says 172.21.15.66:56750) 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 -- 172.21.15.66:0/404018374 learned_addr learned my addr 172.21.15.66:0/404018374 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056dcf0700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0568101680 0x7f05680738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:37.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 -- 172.21.15.66:0/404018374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0568102040 msgr2=0x7f0568073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0568102040 0x7f0568073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 -- 172.21.15.66:0/404018374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0568101680 msgr2=0x7f05680738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0568101680 0x7f05680738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 -- 172.21.15.66:0/404018374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009d00 con 0x7f05681076f0 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.217+0000 7f056e4f1700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 0x7f0568074350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0560007930 tx=0x7f0560007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.218+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056000e040 con 0x7f05681076f0 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.218+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0560004c50 con 0x7f05681076f0 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.218+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600052a0 con 0x7f05681076f0 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.218+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05681a6410 con 0x7f05681076f0 2023-12-11T19:32:37.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.218+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05681a6880 con 0x7f05681076f0 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.219+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0568109b80 con 0x7f05681076f0 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.221+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f056001b970 con 0x7f05681076f0 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.222+0000 7f055effd700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f05540a1590 0x7f05540a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.222+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05600ce8f0 con 0x7f05681076f0 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.223+0000 7f056dcf0700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f05540a1590 0x7f05540a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.223+0000 7f056dcf0700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f05540a1590 0x7f05540a3a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0564005950 tx=0x7f05640058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:37.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.224+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0560097e40 con 0x7f05681076f0 2023-12-11T19:32:37.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.437+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0568074db0 con 0x7f05681076f0 2023-12-11T19:32:37.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.438+0000 7f055effd700 1 -- 172.21.15.66:0/404018374 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f056009b460 con 0x7f05681076f0 2023-12-11T19:32:37.439 INFO:teuthology.orchestra.run.smithi066.stdout:146028888125 2023-12-11T19:32:37.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f05540a1590 msgr2=0x7f05540a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:37.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f05540a1590 0x7f05540a3a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0564005950 tx=0x7f05640058e0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 msgr2=0x7f0568074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 0x7f0568074350 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0560007930 tx=0x7f0560007a10 comp rx=0 tx=0).stop 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 shutdown_connections 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f05540a1590 0x7f05540a3a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0568101680 0x7f05680738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05681076f0 0x7f0568074350 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 --2- 172.21.15.66:0/404018374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0568102040 0x7f0568073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:37.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 >> 172.21.15.66:0/404018374 conn(0x7f05680787c0 msgr2=0x7f0568100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:37.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 shutdown_connections 2023-12-11T19:32:37.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:37.440+0000 7f056ff54700 1 -- 172.21.15.66:0/404018374 wait complete. 2023-12-11T19:32:37.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:37 smithi066 ceph-mon[97920]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:37 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4222385142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-11T19:32:37.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:37 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/404018374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-11T19:32:38.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:37 smithi167 ceph-mon[102750]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:38.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:37 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4222385142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-11T19:32:38.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:37 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/404018374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-11T19:32:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:37 smithi070 ceph-mon[104044]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:37 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4222385142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-11T19:32:38.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:37 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/404018374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-11T19:32:38.329 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574923 for osd.1 2023-12-11T19:32:38.329 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:38.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149762 for osd.3 2023-12-11T19:32:38.391 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 -- 172.21.15.66:0/2394364676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348107620 msgr2=0x7fa348107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 --2- 172.21.15.66:0/2394364676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348107620 0x7fa348107a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa33800dcb0 tx=0x7fa33800f050 comp rx=0 tx=0).stop 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 -- 172.21.15.66:0/2394364676 shutdown_connections 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 --2- 172.21.15.66:0/2394364676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348107620 0x7fa348107a00 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa33800dcb0 tx=0x7fa33800f050 comp rx=0 tx=0).stop 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 --2- 172.21.15.66:0/2394364676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa348069930 0x7fa34810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 --2- 172.21.15.66:0/2394364676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa348068f70 0x7fa3480693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.443+0000 7fa34e264700 1 -- 172.21.15.66:0/2394364676 >> 172.21.15.66:0/2394364676 conn(0x7fa348076570 msgr2=0x7fa348076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.445+0000 7fa34e264700 1 -- 172.21.15.66:0/2394364676 shutdown_connections 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.445+0000 7fa34e264700 1 -- 172.21.15.66:0/2394364676 wait complete. 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.446+0000 7fa34e264700 1 Processor -- start 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 -- start start 2023-12-11T19:32:38.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348068f70 0x7fa34819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa348069930 0x7fa34819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 0x7fa3481a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481a34a0 con 0x7fa3481a2ab0 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481a3610 con 0x7fa348068f70 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.447+0000 7fa34e264700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481a37d0 con 0x7fa348069930 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 0x7fa3481a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa3477fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa348069930 0x7fa34819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 0x7fa3481a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56772/0 (socket says 172.21.15.66:56772) 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 -- 172.21.15.66:0/3702839829 learned_addr learned my addr 172.21.15.66:0/3702839829 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 -- 172.21.15.66:0/3702839829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa348069930 msgr2=0x7fa34819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa348069930 0x7fa34819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 -- 172.21.15.66:0/3702839829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348068f70 msgr2=0x7fa34819e1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348068f70 0x7fa34819e1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.448+0000 7fa34c801700 1 -- 172.21.15.66:0/3702839829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340009d00 con 0x7fa3481a2ab0 2023-12-11T19:32:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.449+0000 7fa34c801700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 0x7fa3481a2f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa34000e3c0 tx=0x7fa34000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.450+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3400092d0 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.450+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33800d930 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.450+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa34000f040 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.450+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340007c60 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.450+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa348195ad0 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.455+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa348109a00 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.457+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa340003a00 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.459+0000 7fa3457fa700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3300a1610 0x7fa3300a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.460+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3400cca90 con 0x7fa3481a2ab0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.460+0000 7fa347fff700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3300a1610 0x7fa3300a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.461+0000 7fa347fff700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3300a1610 0x7fa3300a3ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa338005950 tx=0x7fa3380058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:38.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.467+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa340095fe0 con 0x7fa3481a2ab0 2023-12-11T19:32:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:38.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 -- 172.21.15.66:0/3058947989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 msgr2=0x7fec14107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/3058947989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14107ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec10009990 tx=0x7fec10009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 -- 172.21.15.66:0/3058947989 shutdown_connections 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/3058947989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec14102040 0x7fec1410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/3058947989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14107ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/3058947989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14101680 0x7fec14101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 -- 172.21.15.66:0/3058947989 >> 172.21.15.66:0/3058947989 conn(0x7fec140787c0 msgr2=0x7fec14078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 -- 172.21.15.66:0/3058947989 shutdown_connections 2023-12-11T19:32:38.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.682+0000 7fec1b3a9700 1 -- 172.21.15.66:0/3058947989 wait complete. 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.683+0000 7fec1b3a9700 1 Processor -- start 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.683+0000 7fec1b3a9700 1 -- start start 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec1b3a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec14101680 0x7fec141084b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec1b3a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14102040 0x7fec141089f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec1b3a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14108f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec1b3a9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec141aa140 con 0x7fec141076f0 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec1b3a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec141aa280 con 0x7fec14102040 2023-12-11T19:32:38.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec1b3a9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec141aa3c0 con 0x7fec14101680 2023-12-11T19:32:38.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14108f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56794/0 (socket says 172.21.15.66:56794) 2023-12-11T19:32:38.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 -- 172.21.15.66:0/489023788 learned_addr learned my addr 172.21.15.66:0/489023788 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:38.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec18944700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14102040 0x7fec141089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19145700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec14101680 0x7fec141084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 -- 172.21.15.66:0/489023788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec14101680 msgr2=0x7fec141084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec14101680 0x7fec141084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 -- 172.21.15.66:0/489023788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14102040 msgr2=0x7fec141089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14102040 0x7fec141089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec19946700 1 -- 172.21.15.66:0/489023788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec100095f0 con 0x7fec141076f0 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.684+0000 7fec18944700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14102040 0x7fec141089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.685+0000 7fec19946700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14108f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00efc0 tx=0x7fec0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.685+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c018070 con 0x7fec141076f0 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.685+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec0c00f040 con 0x7fec141076f0 2023-12-11T19:32:38.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.685+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec141aa640 con 0x7fec141076f0 2023-12-11T19:32:38.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.685+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00cbb0 con 0x7fec141076f0 2023-12-11T19:32:38.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.685+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141aab90 con 0x7fec141076f0 2023-12-11T19:32:38.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.687+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec14102d10 con 0x7fec141076f0 2023-12-11T19:32:38.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.689+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fec0c00cd50 con 0x7fec141076f0 2023-12-11T19:32:38.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.691+0000 7fec0a7fc700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec000a1590 0x7fec000a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:38.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.691+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fec0c0d0380 con 0x7fec141076f0 2023-12-11T19:32:38.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.691+0000 7fec19145700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec000a1590 0x7fec000a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:38.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.692+0000 7fec19145700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec000a1590 0x7fec000a3a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fec10006010 tx=0x7fec1000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.695+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fec0c095710 con 0x7fec141076f0 2023-12-11T19:32:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.741+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fa34819f4d0 con 0x7fa3481a2ab0 2023-12-11T19:32:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.741+0000 7fa3457fa700 1 -- 172.21.15.66:0/3702839829 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa340099600 con 0x7fa3481a2ab0 2023-12-11T19:32:38.745 INFO:teuthology.orchestra.run.smithi066.stdout:292057776161 2023-12-11T19:32:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3300a1610 msgr2=0x7fa3300a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3300a1610 0x7fa3300a3ad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa338005950 tx=0x7fa3380058e0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 msgr2=0x7fa3481a2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 0x7fa3481a2f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa34000e3c0 tx=0x7fa34000e780 comp rx=0 tx=0).stop 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 shutdown_connections 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3300a1610 0x7fa3300a3ad0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa348068f70 0x7fa34819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3481a2ab0 0x7fa3481a2f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 --2- 172.21.15.66:0/3702839829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa348069930 0x7fa34819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.744+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 >> 172.21.15.66:0/3702839829 conn(0x7fa348076570 msgr2=0x7fa348106920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.745+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 shutdown_connections 2023-12-11T19:32:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.745+0000 7fa34e264700 1 -- 172.21.15.66:0/3702839829 wait complete. 2023-12-11T19:32:38.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888125 for osd.4 2023-12-11T19:32:38.868 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.951+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fec141113d0 con 0x7fec141076f0 2023-12-11T19:32:38.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.952+0000 7fec0a7fc700 1 -- 172.21.15.66:0/489023788 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fec0c098d30 con 0x7fec141076f0 2023-12-11T19:32:38.953 INFO:teuthology.orchestra.run.smithi066.stdout:313532612636 2023-12-11T19:32:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec000a1590 msgr2=0x7fec000a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec000a1590 0x7fec000a3a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fec10006010 tx=0x7fec1000b260 comp rx=0 tx=0).stop 2023-12-11T19:32:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 msgr2=0x7fec14108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14108f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00efc0 tx=0x7fec0c00c3a0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 shutdown_connections 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fec000a1590 0x7fec000a3a50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fec141076f0 0x7fec14108f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec14101680 0x7fec141084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 --2- 172.21.15.66:0/489023788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec14102040 0x7fec141089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 >> 172.21.15.66:0/489023788 conn(0x7fec140787c0 msgr2=0x7fec14100270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 shutdown_connections 2023-12-11T19:32:38.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:38.955+0000 7fec1b3a9700 1 -- 172.21.15.66:0/489023788 wait complete. 2023-12-11T19:32:39.544 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037798 for osd.8 2023-12-11T19:32:39.545 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 -- 172.21.15.66:0/354950904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c101680 msgr2=0x7f015c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 --2- 172.21.15.66:0/354950904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c101680 0x7f015c101b00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0150009990 tx=0x7f0150009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 -- 172.21.15.66:0/354950904 shutdown_connections 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 --2- 172.21.15.66:0/354950904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c102040 0x7f015c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 --2- 172.21.15.66:0/354950904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c1076f0 0x7f015c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 --2- 172.21.15.66:0/354950904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c101680 0x7f015c101b00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.695+0000 7f0161679700 1 -- 172.21.15.66:0/354950904 >> 172.21.15.66:0/354950904 conn(0x7f015c0787c0 msgr2=0x7f015c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 -- 172.21.15.66:0/354950904 shutdown_connections 2023-12-11T19:32:39.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 -- 172.21.15.66:0/354950904 wait complete. 2023-12-11T19:32:39.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 Processor -- start 2023-12-11T19:32:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 -- start start 2023-12-11T19:32:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c101680 0x7f015c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c102040 0x7f015c073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:39.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 0x7f015c074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1a5cc0 con 0x7f015c1076f0 2023-12-11T19:32:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1a5e30 con 0x7f015c101680 2023-12-11T19:32:39.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f0161679700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1a5ff0 con 0x7f015c102040 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 0x7f015c074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c102040 0x7f015c073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 0x7f015c074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56806/0 (socket says 172.21.15.66:56806) 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c102040 0x7f015c073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:60430/0 (socket says 172.21.15.66:60430) 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 -- 172.21.15.66:0/3399936377 learned_addr learned my addr 172.21.15.66:0/3399936377 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015affd700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c101680 0x7f015c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 -- 172.21.15.66:0/3399936377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c102040 msgr2=0x7f015c073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c102040 0x7f015c073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 -- 172.21.15.66:0/3399936377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c101680 msgr2=0x7f015c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c101680 0x7f015c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 -- 172.21.15.66:0/3399936377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500095f0 con 0x7f015c1076f0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015affd700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c101680 0x7f015c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.696+0000 7f015b7fe700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 0x7f015c074350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f015400efc0 tx=0x7f015400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.702+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154018070 con 0x7f015c1076f0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.702+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015400f040 con 0x7f015c1076f0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.702+0000 7f0161679700 1 -- 172.21.15.66:0/3399936377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c1a6270 con 0x7f015c1076f0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.702+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400cb90 con 0x7f015c1076f0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.702+0000 7f0161679700 1 -- 172.21.15.66:0/3399936377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c1a67c0 con 0x7f015c1076f0 2023-12-11T19:32:39.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.702+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c102d10 con 0x7f015c1076f0 2023-12-11T19:32:39.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.704+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f015400cd30 con 0x7f015c1076f0 2023-12-11T19:32:39.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.704+0000 7f0143fff700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01440a1590 0x7f01440a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:39.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.704+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f01540cc5d0 con 0x7f015c1076f0 2023-12-11T19:32:39.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.704+0000 7f015affd700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01440a1590 0x7f01440a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:39.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.704+0000 7f015affd700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01440a1590 0x7f01440a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f014c009cb0 tx=0x7f014c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:39.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.711+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0154095b60 con 0x7f015c1076f0 2023-12-11T19:32:39.791 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:39 smithi066 ceph-mon[97920]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:39.791 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:39 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3702839829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-11T19:32:39.791 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:39 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/489023788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-11T19:32:39.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.945+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f015c074d60 con 0x7f015c1076f0 2023-12-11T19:32:39.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.945+0000 7f0143fff700 1 -- 172.21.15.66:0/3399936377 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0154099180 con 0x7f015c1076f0 2023-12-11T19:32:39.947 INFO:teuthology.orchestra.run.smithi066.stdout:201863462963 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.947+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01440a1590 msgr2=0x7f01440a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.947+0000 7f0141ffb700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01440a1590 0x7f01440a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f014c009cb0 tx=0x7f014c009260 comp rx=0 tx=0).stop 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.947+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 msgr2=0x7f015c074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 0x7f015c074350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f015400efc0 tx=0x7f015400c3a0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 shutdown_connections 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01440a1590 0x7f01440a3a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f015c101680 0x7f015c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f015c1076f0 0x7f015c074350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 --2- 172.21.15.66:0/3399936377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015c102040 0x7f015c073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:39.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 >> 172.21.15.66:0/3399936377 conn(0x7f015c0787c0 msgr2=0x7f015c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:39.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 shutdown_connections 2023-12-11T19:32:39.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:39.948+0000 7f0141ffb700 1 -- 172.21.15.66:0/3399936377 wait complete. 2023-12-11T19:32:40.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:39 smithi167 ceph-mon[102750]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:40.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:39 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3702839829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-11T19:32:40.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:39 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/489023788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-11T19:32:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:39 smithi070 ceph-mon[104044]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:32:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:39 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3702839829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-11T19:32:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:39 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/489023788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-11T19:32:40.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 -- 172.21.15.66:0/31687317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c24104160 msgr2=0x7f6c24110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 --2- 172.21.15.66:0/31687317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c24104160 0x7f6c24110670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b0a0 tx=0x7f6c1c00b3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 -- 172.21.15.66:0/31687317 shutdown_connections 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 --2- 172.21.15.66:0/31687317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c24104160 0x7f6c24110670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 --2- 172.21.15.66:0/31687317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c241097a0 0x7f6c24109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 --2- 172.21.15.66:0/31687317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c241037a0 0x7f6c24103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 -- 172.21.15.66:0/31687317 >> 172.21.15.66:0/31687317 conn(0x7f6c240ff290 msgr2=0x7f6c241016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 -- 172.21.15.66:0/31687317 shutdown_connections 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.657+0000 7f6c295e7700 1 -- 172.21.15.66:0/31687317 wait complete. 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.659+0000 7f6c295e7700 1 Processor -- start 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.659+0000 7f6c295e7700 1 -- start start 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c295e7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c241037a0 0x7f6c2419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c295e7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c24104160 0x7f6c2419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c295e7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 0x7f6c24199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c295e7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419a120 con 0x7f6c241097a0 2023-12-11T19:32:40.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c295e7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419a290 con 0x7f6c24104160 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c295e7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419a450 con 0x7f6c241037a0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 0x7f6c24199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 0x7f6c24199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56820/0 (socket says 172.21.15.66:56820) 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 -- 172.21.15.66:0/1846987369 learned_addr learned my addr 172.21.15.66:0/1846987369 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c22ffd700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c241037a0 0x7f6c2419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c227fc700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c24104160 0x7f6c2419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 -- 172.21.15.66:0/1846987369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c241037a0 msgr2=0x7f6c2419f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c241037a0 0x7f6c2419f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 -- 172.21.15.66:0/1846987369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c24104160 msgr2=0x7f6c2419fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c24104160 0x7f6c2419fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.660+0000 7f6c237fe700 1 -- 172.21.15.66:0/1846987369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009d00 con 0x7f6c241097a0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.661+0000 7f6c237fe700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 0x7f6c24199ac0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b070 tx=0x7f6c1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.661+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00e040 con 0x7f6c241097a0 2023-12-11T19:32:40.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.661+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2419a6d0 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.661+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c1c003e90 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.661+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c005040 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.661+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c241a6970 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.662+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2410bdc0 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.664+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f6c1c01d610 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.665+0000 7f6c0bfff700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6c0c0a1510 0x7f6c0c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.666+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c1c0cdfc0 con 0x7f6c241097a0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.666+0000 7f6c22ffd700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6c0c0a1510 0x7f6c0c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.666+0000 7f6c22ffd700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6c0c0a1510 0x7f6c0c0a39d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009be0 tx=0x7f6c14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:40.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.669+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6c1c097510 con 0x7f6c241097a0 2023-12-11T19:32:40.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.885+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6c24104ee0 con 0x7f6c241097a0 2023-12-11T19:32:40.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.885+0000 7f6c0bfff700 1 -- 172.21.15.66:0/1846987369 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c1c011390 con 0x7f6c241097a0 2023-12-11T19:32:40.887 INFO:teuthology.orchestra.run.smithi066.stdout:77309411401 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6c0c0a1510 msgr2=0x7f6c0c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6c0c0a1510 0x7f6c0c0a39d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009be0 tx=0x7f6c14009260 comp rx=0 tx=0).stop 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 msgr2=0x7f6c24199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 0x7f6c24199ac0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b070 tx=0x7f6c1c00be20 comp rx=0 tx=0).stop 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 shutdown_connections 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6c0c0a1510 0x7f6c0c0a39d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c241097a0 0x7f6c24199ac0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c241037a0 0x7f6c2419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 --2- 172.21.15.66:0/1846987369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c24104160 0x7f6c2419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:40.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 >> 172.21.15.66:0/1846987369 conn(0x7f6c240ff290 msgr2=0x7f6c240ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:40.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 shutdown_connections 2023-12-11T19:32:40.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:40.888+0000 7f6c295e7700 1 -- 172.21.15.66:0/1846987369 wait complete. 2023-12-11T19:32:40.911 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:40 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3399936377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-11T19:32:41.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:40 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3399936377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-11T19:32:41.105 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462963 for osd.6 2023-12-11T19:32:41.105 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:41.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:40 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3399936377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-11T19:32:41.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776161 for osd.9 2023-12-11T19:32:41.335 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:41.610 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411401 for osd.2 2023-12-11T19:32:41.610 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:41.737 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612636 for osd.10 2023-12-11T19:32:41.737 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:41.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:41 smithi066 ceph-mon[97920]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-11T19:32:41.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:41 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1846987369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-11T19:32:42.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:41 smithi167 ceph-mon[102750]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-11T19:32:42.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:41 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1846987369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-11T19:32:42.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:41 smithi070 ceph-mon[104044]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-11T19:32:42.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:41 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1846987369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-11T19:32:42.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.577+0000 7f02dc105700 1 -- 172.21.15.66:0/3234435189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4068f70 msgr2=0x7f02d40693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.577+0000 7f02dc105700 1 --2- 172.21.15.66:0/3234435189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4068f70 0x7f02d40693f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02c40099c0 tx=0x7f02c4009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 -- 172.21.15.66:0/3234435189 shutdown_connections 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 --2- 172.21.15.66:0/3234435189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4069930 0x7f02d410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 --2- 172.21.15.66:0/3234435189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d4107620 0x7f02d4107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 --2- 172.21.15.66:0/3234435189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4068f70 0x7f02d40693f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 -- 172.21.15.66:0/3234435189 >> 172.21.15.66:0/3234435189 conn(0x7f02d4076570 msgr2=0x7f02d4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 -- 172.21.15.66:0/3234435189 shutdown_connections 2023-12-11T19:32:42.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.578+0000 7f02dc105700 1 -- 172.21.15.66:0/3234435189 wait complete. 2023-12-11T19:32:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 Processor -- start 2023-12-11T19:32:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 -- start start 2023-12-11T19:32:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 0x7f02d419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d4069930 0x7f02d419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4107620 0x7f02d4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4197da0 con 0x7f02d4107620 2023-12-11T19:32:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4197f10 con 0x7f02d4068f70 2023-12-11T19:32:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02dc105700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41980d0 con 0x7f02d4069930 2023-12-11T19:32:42.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.580+0000 7f02da6a2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4107620 0x7f02d4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 0x7f02d419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 0x7f02d419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:47256/0 (socket says 172.21.15.66:47256) 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 -- 172.21.15.66:0/2224866772 learned_addr learned my addr 172.21.15.66:0/2224866772 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d96a0700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d4069930 0x7f02d419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 -- 172.21.15.66:0/2224866772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d4069930 msgr2=0x7f02d419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d4069930 0x7f02d419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 -- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4107620 msgr2=0x7f02d4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4107620 0x7f02d4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 -- 172.21.15.66:0/2224866772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40095f0 con 0x7f02d4068f70 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02da6a2700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4107620 0x7f02d4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:42.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.581+0000 7f02d9ea1700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 0x7f02d419d600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02d000d980 tx=0x7f02d000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:42.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.582+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0004220 con 0x7f02d4068f70 2023-12-11T19:32:42.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.582+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d41983b0 con 0x7f02d4068f70 2023-12-11T19:32:42.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.582+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02d0007640 con 0x7f02d4068f70 2023-12-11T19:32:42.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.582+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0005660 con 0x7f02d4068f70 2023-12-11T19:32:42.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.582+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d41a45f0 con 0x7f02d4068f70 2023-12-11T19:32:42.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.583+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d4109a20 con 0x7f02d4068f70 2023-12-11T19:32:42.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.585+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f02d0004750 con 0x7f02d4068f70 2023-12-11T19:32:42.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.587+0000 7f02caffd700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f02c00a1590 0x7f02c00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.587+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02d00ccd80 con 0x7f02d4068f70 2023-12-11T19:32:42.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.587+0000 7f02d96a0700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f02c00a1590 0x7f02c00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.588+0000 7f02d96a0700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f02c00a1590 0x7f02c00a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009990 tx=0x7f02c40052e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:42.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.589+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f02d0096240 con 0x7f02d4068f70 2023-12-11T19:32:42.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 -- 172.21.15.66:0/2718568847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 msgr2=0x7f06ac1147d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 --2- 172.21.15.66:0/2718568847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1147d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f06a400b0a0 tx=0x7f06a400b3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 -- 172.21.15.66:0/2718568847 shutdown_connections 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 --2- 172.21.15.66:0/2718568847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1147d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 --2- 172.21.15.66:0/2718568847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06ac073970 0x7f06ac073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 --2- 172.21.15.66:0/2718568847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06ac074290 0x7f06ac111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 -- 172.21.15.66:0/2718568847 >> 172.21.15.66:0/2718568847 conn(0x7f06ac0fcf00 msgr2=0x7f06ac0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 -- 172.21.15.66:0/2718568847 shutdown_connections 2023-12-11T19:32:42.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.691+0000 7f06b176c700 1 -- 172.21.15.66:0/2718568847 wait complete. 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.692+0000 7f06b176c700 1 Processor -- start 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.692+0000 7f06b176c700 1 -- start start 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.692+0000 7f06b176c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06ac073970 0x7f06ac19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b176c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06ac074290 0x7f06ac19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b176c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b176c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a2dc0 con 0x7f06ac1123e0 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b176c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a2f30 con 0x7f06ac074290 2023-12-11T19:32:42.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b176c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1a30f0 con 0x7f06ac073970 2023-12-11T19:32:42.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1a27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49488/0 (socket says 172.21.15.66:49488) 2023-12-11T19:32:42.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 -- 172.21.15.66:0/2748776974 learned_addr learned my addr 172.21.15.66:0/2748776974 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:42.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06abfff700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06ac073970 0x7f06ac19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06ab7fe700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06ac074290 0x7f06ac19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 -- 172.21.15.66:0/2748776974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06ac073970 msgr2=0x7f06ac19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06ac073970 0x7f06ac19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 -- 172.21.15.66:0/2748776974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06ac074290 msgr2=0x7f06ac19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06ac074290 0x7f06ac19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.693+0000 7f06b0f6b700 1 -- 172.21.15.66:0/2748776974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009d00 con 0x7f06ac1123e0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.694+0000 7f06b0f6b700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1a27b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06a400b070 tx=0x7f06a4007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.694+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a400e040 con 0x7f06ac1123e0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.694+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06a400be20 con 0x7f06ac1123e0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.694+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4005040 con 0x7f06ac1123e0 2023-12-11T19:32:42.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.694+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac1a3370 con 0x7f06ac1123e0 2023-12-11T19:32:42.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.694+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac1aac30 con 0x7f06ac1123e0 2023-12-11T19:32:42.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.695+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac1048e0 con 0x7f06ac1123e0 2023-12-11T19:32:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.697+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f06a401b450 con 0x7f06ac1123e0 2023-12-11T19:32:42.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.699+0000 7f06a97fa700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06940a1590 0x7f06940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:42.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.700+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06a40ce330 con 0x7f06ac1123e0 2023-12-11T19:32:42.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.700+0000 7f06abfff700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06940a1590 0x7f06940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.701+0000 7f06abfff700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06940a1590 0x7f06940a3a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f069c009bf0 tx=0x7f069c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:42.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.704+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f06a4097880 con 0x7f06ac1123e0 2023-12-11T19:32:42.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.817+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f02d4198c00 con 0x7f02d4068f70 2023-12-11T19:32:42.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.817+0000 7f02caffd700 1 -- 172.21.15.66:0/2224866772 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f02d0016070 con 0x7f02d4068f70 2023-12-11T19:32:42.819 INFO:teuthology.orchestra.run.smithi066.stdout:347892350999 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f02c00a1590 msgr2=0x7f02c00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f02c00a1590 0x7f02c00a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009990 tx=0x7f02c40052e0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 msgr2=0x7f02d419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 0x7f02d419d600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02d000d980 tx=0x7f02d000dc90 comp rx=0 tx=0).stop 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 shutdown_connections 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f02c00a1590 0x7f02c00a3a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d4068f70 0x7f02d419d600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d4107620 0x7f02d4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 --2- 172.21.15.66:0/2224866772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d4069930 0x7f02d419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 >> 172.21.15.66:0/2224866772 conn(0x7f02d4076570 msgr2=0x7f02d40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 shutdown_connections 2023-12-11T19:32:42.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.820+0000 7f02dc105700 1 -- 172.21.15.66:0/2224866772 wait complete. 2023-12-11T19:32:42.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.939+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06ac19f1d0 con 0x7f06ac1123e0 2023-12-11T19:32:42.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.939+0000 7f06a97fa700 1 -- 172.21.15.66:0/2748776974 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06a409aea0 con 0x7f06ac1123e0 2023-12-11T19:32:42.941 INFO:teuthology.orchestra.run.smithi066.stdout:34359738452 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.942+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06940a1590 msgr2=0x7f06940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06940a1590 0x7f06940a3a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f069c009bf0 tx=0x7f069c009260 comp rx=0 tx=0).stop 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 msgr2=0x7f06ac1a27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1a27b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06a400b070 tx=0x7f06a4007d70 comp rx=0 tx=0).stop 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 shutdown_connections 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06940a1590 0x7f06940a3a50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06ac1123e0 0x7f06ac1a27b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06ac073970 0x7f06ac19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 --2- 172.21.15.66:0/2748776974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06ac074290 0x7f06ac19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:42.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 >> 172.21.15.66:0/2748776974 conn(0x7f06ac0fcf00 msgr2=0x7f06ac0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:42.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 shutdown_connections 2023-12-11T19:32:42.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:42.943+0000 7f06b176c700 1 -- 172.21.15.66:0/2748776974 wait complete. 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 -- 172.21.15.66:0/3893151703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc074120 msgr2=0x7f6dfc10daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 --2- 172.21.15.66:0/3893151703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc074120 0x7f6dfc10daf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009990 tx=0x7f6dec009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 -- 172.21.15.66:0/3893151703 shutdown_connections 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 --2- 172.21.15.66:0/3893151703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dfc10e030 0x7f6dfc110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 --2- 172.21.15.66:0/3893151703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dfc073800 0x7f6dfc073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 --2- 172.21.15.66:0/3893151703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc074120 0x7f6dfc10daf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 -- 172.21.15.66:0/3893151703 >> 172.21.15.66:0/3893151703 conn(0x7f6dfc0787c0 msgr2=0x7f6dfc078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:43.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 -- 172.21.15.66:0/3893151703 shutdown_connections 2023-12-11T19:32:43.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.138+0000 7f6e03b58700 1 -- 172.21.15.66:0/3893151703 wait complete. 2023-12-11T19:32:43.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.139+0000 7f6e03b58700 1 Processor -- start 2023-12-11T19:32:43.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 -- start start 2023-12-11T19:32:43.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dfc073800 0x7f6dfc19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:43.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dfc074120 0x7f6dfc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:43.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 0x7f6dfc199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:43.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc19a070 con 0x7f6dfc10e030 2023-12-11T19:32:43.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc19a1e0 con 0x7f6dfc074120 2023-12-11T19:32:43.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e03b58700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc19a3a0 con 0x7f6dfc073800 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e020f5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 0x7f6dfc199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e020f5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 0x7f6dfc199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49506/0 (socket says 172.21.15.66:49506) 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e020f5700 1 -- 172.21.15.66:0/1262288235 learned_addr learned my addr 172.21.15.66:0/1262288235 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e010f3700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dfc074120 0x7f6dfc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.140+0000 7f6e018f4700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dfc073800 0x7f6dfc19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e020f5700 1 -- 172.21.15.66:0/1262288235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dfc073800 msgr2=0x7f6dfc19f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e020f5700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dfc073800 0x7f6dfc19f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e020f5700 1 -- 172.21.15.66:0/1262288235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dfc074120 msgr2=0x7f6dfc19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e020f5700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dfc074120 0x7f6dfc19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e020f5700 1 -- 172.21.15.66:0/1262288235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0095f0 con 0x7f6dfc10e030 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e020f5700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 0x7f6dfc199a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6df4007ee0 tx=0x7f6df400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400ddd0 con 0x7f6dfc10e030 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6df400f040 con 0x7f6dfc10e030 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4005970 con 0x7f6dfc10e030 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc19a680 con 0x7f6dfc10e030 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.141+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc1a68c0 con 0x7f6dfc10e030 2023-12-11T19:32:43.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.143+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc109240 con 0x7f6dfc10e030 2023-12-11T19:32:43.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.145+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f6df4004fb0 con 0x7f6dfc10e030 2023-12-11T19:32:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.146+0000 7f6df2ffd700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6de80a1590 0x7f6de80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:43.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.146+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6df40cc660 con 0x7f6dfc10e030 2023-12-11T19:32:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.146+0000 7f6e018f4700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6de80a1590 0x7f6de80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:43.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.147+0000 7f6e018f4700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6de80a1590 0x7f6de80a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6df800bbd0 tx=0x7f6df800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:43.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.150+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6df404f2f0 con 0x7f6dfc10e030 2023-12-11T19:32:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.366+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6dfc074e90 con 0x7f6dfc10e030 2023-12-11T19:32:43.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.367+0000 7f6df2ffd700 1 -- 172.21.15.66:0/1262288235 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6df4095b20 con 0x7f6dfc10e030 2023-12-11T19:32:43.368 INFO:teuthology.orchestra.run.smithi066.stdout:167503724601 2023-12-11T19:32:43.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6de80a1590 msgr2=0x7f6de80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:43.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6de80a1590 0x7f6de80a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6df800bbd0 tx=0x7f6df800b5b0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 msgr2=0x7f6dfc199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 0x7f6dfc199a10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6df4007ee0 tx=0x7f6df400d3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 shutdown_connections 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6de80a1590 0x7f6de80a3a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6dfc10e030 0x7f6dfc199a10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6dfc073800 0x7f6dfc19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 --2- 172.21.15.66:0/1262288235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dfc074120 0x7f6dfc19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 >> 172.21.15.66:0/1262288235 conn(0x7f6dfc0787c0 msgr2=0x7f6dfc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 shutdown_connections 2023-12-11T19:32:43.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:43.369+0000 7f6e03b58700 1 -- 172.21.15.66:0/1262288235 wait complete. 2023-12-11T19:32:43.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350997 got 347892350999 for osd.11 2023-12-11T19:32:43.479 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:43.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724598 got 167503724601 for osd.5 2023-12-11T19:32:43.840 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:43.927 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:43 smithi066 ceph-mon[97920]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-12-11T19:32:43.928 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:43 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2224866772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-11T19:32:43.928 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:43 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2748776974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-11T19:32:43.928 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:43 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1262288235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-11T19:32:44.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:43 smithi167 ceph-mon[102750]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-12-11T19:32:44.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:43 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2224866772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-11T19:32:44.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:43 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2748776974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-11T19:32:44.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:43 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1262288235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-11T19:32:44.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738452 for osd.0 2023-12-11T19:32:44.079 DEBUG:teuthology.parallel:result is None 2023-12-11T19:32:44.079 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-11T19:32:44.079 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph pg dump --format=json 2023-12-11T19:32:44.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:43 smithi070 ceph-mon[104044]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-12-11T19:32:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:43 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2224866772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-11T19:32:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:43 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2748776974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-11T19:32:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:43 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1262288235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-11T19:32:44.887 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:45.800 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:45 smithi066 ceph-mon[97920]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:46.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:45 smithi167 ceph-mon[102750]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:45 smithi070 ceph-mon[104044]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.510+0000 7f9ddc13f700 1 -- 172.21.15.66:0/4061621117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd41076f0 msgr2=0x7f9dd4107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.510+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/4061621117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd41076f0 0x7f9dd4107ad0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009990 tx=0x7f9dd0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 -- 172.21.15.66:0/4061621117 shutdown_connections 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/4061621117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dd4102040 0x7f9dd410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/4061621117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd41076f0 0x7f9dd4107ad0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/4061621117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd4101680 0x7f9dd4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 -- 172.21.15.66:0/4061621117 >> 172.21.15.66:0/4061621117 conn(0x7f9dd40787c0 msgr2=0x7f9dd4078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 -- 172.21.15.66:0/4061621117 shutdown_connections 2023-12-11T19:32:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.511+0000 7f9ddc13f700 1 -- 172.21.15.66:0/4061621117 wait complete. 2023-12-11T19:32:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 Processor -- start 2023-12-11T19:32:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 -- start start 2023-12-11T19:32:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dd4101680 0x7f9dd419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 0x7f9dd419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd41076f0 0x7f9dd41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd419a030 con 0x7f9dd4102040 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd419a1a0 con 0x7f9dd4101680 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.512+0000 7f9ddc13f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd419a360 con 0x7f9dd41076f0 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 0x7f9dd419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd9edb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dd4101680 0x7f9dd419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 0x7f9dd419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49522/0 (socket says 172.21.15.66:49522) 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 -- 172.21.15.66:0/978605071 learned_addr learned my addr 172.21.15.66:0/978605071 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dda6dc700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd41076f0 0x7f9dd41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 -- 172.21.15.66:0/978605071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd41076f0 msgr2=0x7f9dd41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd41076f0 0x7f9dd41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 -- 172.21.15.66:0/978605071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dd4101680 msgr2=0x7f9dd419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dd4101680 0x7f9dd419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.513+0000 7f9dd96da700 1 -- 172.21.15.66:0/978605071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd00095f0 con 0x7f9dd4102040 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.514+0000 7f9dd96da700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 0x7f9dd419fdd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400b530 tx=0x7f9dc400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.514+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4011830 con 0x7f9dd4102040 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.514+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dc4011e20 con 0x7f9dd4102040 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.514+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400f2e0 con 0x7f9dd4102040 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.514+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd419a640 con 0x7f9dd4102040 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.514+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd41a6900 con 0x7f9dd4102040 2023-12-11T19:32:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.515+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd4104940 con 0x7f9dd4102040 2023-12-11T19:32:46.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.517+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9dc4011990 con 0x7f9dd4102040 2023-12-11T19:32:46.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.518+0000 7f9dcaffd700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9dc00a1590 0x7f9dc00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:46.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.519+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9dc40cc8a0 con 0x7f9dd4102040 2023-12-11T19:32:46.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.519+0000 7f9dd9edb700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9dc00a1590 0x7f9dc00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:46.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.519+0000 7f9dd9edb700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9dc00a1590 0x7f9dc00a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b5c0 tx=0x7f9dd0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:46.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.523+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9dc4095e70 con 0x7f9dd4102040 2023-12-11T19:32:46.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.729+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dd41114d0 con 0x7f9dc00a1590 2023-12-11T19:32:46.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.741+0000 7f9dcaffd700 1 -- 172.21.15.66:0/978605071 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339892 (secure 0 0 0) 0x7f9dd41114d0 con 0x7f9dc00a1590 2023-12-11T19:32:46.747 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:46.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.745+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9dc00a1590 msgr2=0x7f9dc00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:46.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.745+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9dc00a1590 0x7f9dc00a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b5c0 tx=0x7f9dd0005fb0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.745+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 msgr2=0x7f9dd419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.745+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 0x7f9dd419fdd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400b530 tx=0x7f9dc400b840 comp rx=0 tx=0).stop 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 shutdown_connections 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9dc00a1590 0x7f9dc00a3a50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9dd4101680 0x7f9dd419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd41076f0 0x7f9dd41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 --2- 172.21.15.66:0/978605071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dd4102040 0x7f9dd419fdd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 >> 172.21.15.66:0/978605071 conn(0x7f9dd40787c0 msgr2=0x7f9dd410cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 shutdown_connections 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:46.746+0000 7f9ddc13f700 1 -- 172.21.15.66:0/978605071 wait complete. 2023-12-11T19:32:46.751 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:32:47.287 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2023-12-11T19:32:46.298531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657823,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":609,"num_read_kb":429,"num_write":413,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1183040,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336032,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605792,"statfs":{"total":1151940427776,"available":1151596331008,"internally_reserved":0,"allocated":12066816,"data_stored":7475458,"data_compressed":183741,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003126"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583470+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:32:10.583470+0000","last_peered":"2023-12-11T19:32:10.583470+0000","last_clean":"2023-12-11T19:32:10.583470+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:32:10.583470+0000","last_undegraded":"2023-12-11T19:32:10.583470+0000","last_fullsized":"2023-12-11T19:32:10.583470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579009+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:32:10.579009+0000","last_peered":"2023-12-11T19:32:10.579009+0000","last_clean":"2023-12-11T19:32:10.579009+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:32:10.579009+0000","last_undegraded":"2023-12-11T19:32:10.579009+0000","last_fullsized":"2023-12-11T19:32:10.579009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579128+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:32:10.579128+0000","last_peered":"2023-12-11T19:32:10.579128+0000","last_clean":"2023-12-11T19:32:10.579128+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:32:10.579128+0000","last_undegraded":"2023-12-11T19:32:10.579128+0000","last_fullsized":"2023-12-11T19:32:10.579128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582246+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:32:10.582246+0000","last_peered":"2023-12-11T19:32:10.582246+0000","last_clean":"2023-12-11T19:32:10.582246+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:32:10.582246+0000","last_undegraded":"2023-12-11T19:32:10.582246+0000","last_fullsized":"2023-12-11T19:32:10.582246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584906+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:32:10.584906+0000","last_peered":"2023-12-11T19:32:10.584906+0000","last_clean":"2023-12-11T19:32:10.584906+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:32:10.584906+0000","last_undegraded":"2023-12-11T19:32:10.584906+0000","last_fullsized":"2023-12-11T19:32:10.584906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.585539+0000","last_change":"2023-12-11T19:32:11.585539+0000","last_active":"2023-12-11T19:32:11.585539+0000","last_peered":"2023-12-11T19:32:11.585539+0000","last_clean":"2023-12-11T19:32:11.585539+0000","last_became_active":"2023-12-11T19:32:11.585087+0000","last_became_peered":"2023-12-11T19:32:11.585087+0000","last_unstale":"2023-12-11T19:32:11.585539+0000","last_undegraded":"2023-12-11T19:32:11.585539+0000","last_fullsized":"2023-12-11T19:32:11.585539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:23.076166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578059+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:32:10.578059+0000","last_peered":"2023-12-11T19:32:10.578059+0000","last_clean":"2023-12-11T19:32:10.578059+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:32:10.578059+0000","last_undegraded":"2023-12-11T19:32:10.578059+0000","last_fullsized":"2023-12-11T19:32:10.578059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585017+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:32:10.585017+0000","last_peered":"2023-12-11T19:32:10.585017+0000","last_clean":"2023-12-11T19:32:10.585017+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:32:10.585017+0000","last_undegraded":"2023-12-11T19:32:10.585017+0000","last_fullsized":"2023-12-11T19:32:10.585017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583595+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:32:10.583595+0000","last_peered":"2023-12-11T19:32:10.583595+0000","last_clean":"2023-12-11T19:32:10.583595+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:32:10.583595+0000","last_undegraded":"2023-12-11T19:32:10.583595+0000","last_fullsized":"2023-12-11T19:32:10.583595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585811+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:32:10.585811+0000","last_peered":"2023-12-11T19:32:10.585811+0000","last_clean":"2023-12-11T19:32:10.585811+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:32:10.585811+0000","last_undegraded":"2023-12-11T19:32:10.585811+0000","last_fullsized":"2023-12-11T19:32:10.585811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578497+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:32:10.578497+0000","last_peered":"2023-12-11T19:32:10.578497+0000","last_clean":"2023-12-11T19:32:10.578497+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:32:10.578497+0000","last_undegraded":"2023-12-11T19:32:10.578497+0000","last_fullsized":"2023-12-11T19:32:10.578497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582199+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:32:10.582199+0000","last_peered":"2023-12-11T19:32:10.582199+0000","last_clean":"2023-12-11T19:32:10.582199+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:32:10.582199+0000","last_undegraded":"2023-12-11T19:32:10.582199+0000","last_fullsized":"2023-12-11T19:32:10.582199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582043+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:32:10.582043+0000","last_peered":"2023-12-11T19:32:10.582043+0000","last_clean":"2023-12-11T19:32:10.582043+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:32:10.582043+0000","last_undegraded":"2023-12-11T19:32:10.582043+0000","last_fullsized":"2023-12-11T19:32:10.582043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578129+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:32:10.578129+0000","last_peered":"2023-12-11T19:32:10.578129+0000","last_clean":"2023-12-11T19:32:10.578129+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:32:10.578129+0000","last_undegraded":"2023-12-11T19:32:10.578129+0000","last_fullsized":"2023-12-11T19:32:10.578129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.577960+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:32:10.577960+0000","last_peered":"2023-12-11T19:32:10.577960+0000","last_clean":"2023-12-11T19:32:10.577960+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:32:10.577960+0000","last_undegraded":"2023-12-11T19:32:10.577960+0000","last_fullsized":"2023-12-11T19:32:10.577960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581063+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:32:10.581063+0000","last_peered":"2023-12-11T19:32:10.581063+0000","last_clean":"2023-12-11T19:32:10.581063+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:32:10.581063+0000","last_undegraded":"2023-12-11T19:32:10.581063+0000","last_fullsized":"2023-12-11T19:32:10.581063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584127+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:32:10.584127+0000","last_peered":"2023-12-11T19:32:10.584127+0000","last_clean":"2023-12-11T19:32:10.584127+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:32:10.584127+0000","last_undegraded":"2023-12-11T19:32:10.584127+0000","last_fullsized":"2023-12-11T19:32:10.584127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584169+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:32:10.584169+0000","last_peered":"2023-12-11T19:32:10.584169+0000","last_clean":"2023-12-11T19:32:10.584169+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:32:10.584169+0000","last_undegraded":"2023-12-11T19:32:10.584169+0000","last_fullsized":"2023-12-11T19:32:10.584169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586383+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:32:10.586383+0000","last_peered":"2023-12-11T19:32:10.586383+0000","last_clean":"2023-12-11T19:32:10.586383+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:32:10.586383+0000","last_undegraded":"2023-12-11T19:32:10.586383+0000","last_fullsized":"2023-12-11T19:32:10.586383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:42.624082+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:32:42.624082+0000","last_peered":"2023-12-11T19:32:42.624082+0000","last_clean":"2023-12-11T19:32:42.624082+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:32:42.624082+0000","last_undegraded":"2023-12-11T19:32:42.624082+0000","last_fullsized":"2023-12-11T19:32:42.624082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586670+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:32:10.586670+0000","last_peered":"2023-12-11T19:32:10.586670+0000","last_clean":"2023-12-11T19:32:10.586670+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:32:10.586670+0000","last_undegraded":"2023-12-11T19:32:10.586670+0000","last_fullsized":"2023-12-11T19:32:10.586670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586855+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:32:10.586855+0000","last_peered":"2023-12-11T19:32:10.586855+0000","last_clean":"2023-12-11T19:32:10.586855+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:32:10.586855+0000","last_undegraded":"2023-12-11T19:32:10.586855+0000","last_fullsized":"2023-12-11T19:32:10.586855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579198+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:32:10.579198+0000","last_peered":"2023-12-11T19:32:10.579198+0000","last_clean":"2023-12-11T19:32:10.579198+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:32:10.579198+0000","last_undegraded":"2023-12-11T19:32:10.579198+0000","last_fullsized":"2023-12-11T19:32:10.579198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585009+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:32:10.585009+0000","last_peered":"2023-12-11T19:32:10.585009+0000","last_clean":"2023-12-11T19:32:10.585009+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:32:10.585009+0000","last_undegraded":"2023-12-11T19:32:10.585009+0000","last_fullsized":"2023-12-11T19:32:10.585009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582475+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:32:10.582475+0000","last_peered":"2023-12-11T19:32:10.582475+0000","last_clean":"2023-12-11T19:32:10.582475+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:32:10.582475+0000","last_undegraded":"2023-12-11T19:32:10.582475+0000","last_fullsized":"2023-12-11T19:32:10.582475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584594+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:32:10.584594+0000","last_peered":"2023-12-11T19:32:10.584594+0000","last_clean":"2023-12-11T19:32:10.584594+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:32:10.584594+0000","last_undegraded":"2023-12-11T19:32:10.584594+0000","last_fullsized":"2023-12-11T19:32:10.584594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581728+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:32:10.581728+0000","last_peered":"2023-12-11T19:32:10.581728+0000","last_clean":"2023-12-11T19:32:10.581728+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:32:10.581728+0000","last_undegraded":"2023-12-11T19:32:10.581728+0000","last_fullsized":"2023-12-11T19:32:10.581728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:42.643702+0000","last_change":"2023-12-11T19:32:11.601552+0000","last_active":"2023-12-11T19:32:42.643702+0000","last_peered":"2023-12-11T19:32:42.643702+0000","last_clean":"2023-12-11T19:32:42.643702+0000","last_became_active":"2023-12-11T19:32:11.593828+0000","last_became_peered":"2023-12-11T19:32:11.593828+0000","last_unstale":"2023-12-11T19:32:42.643702+0000","last_undegraded":"2023-12-11T19:32:42.643702+0000","last_fullsized":"2023-12-11T19:32:42.643702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:53.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583003+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:32:10.583003+0000","last_peered":"2023-12-11T19:32:10.583003+0000","last_clean":"2023-12-11T19:32:10.583003+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:32:10.583003+0000","last_undegraded":"2023-12-11T19:32:10.583003+0000","last_fullsized":"2023-12-11T19:32:10.583003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586092+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:32:10.586092+0000","last_peered":"2023-12-11T19:32:10.586092+0000","last_clean":"2023-12-11T19:32:10.586092+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:32:10.586092+0000","last_undegraded":"2023-12-11T19:32:10.586092+0000","last_fullsized":"2023-12-11T19:32:10.586092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582047+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:32:10.582047+0000","last_peered":"2023-12-11T19:32:10.582047+0000","last_clean":"2023-12-11T19:32:10.582047+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:32:10.582047+0000","last_undegraded":"2023-12-11T19:32:10.582047+0000","last_fullsized":"2023-12-11T19:32:10.582047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581195+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:32:10.581195+0000","last_peered":"2023-12-11T19:32:10.581195+0000","last_clean":"2023-12-11T19:32:10.581195+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:32:10.581195+0000","last_undegraded":"2023-12-11T19:32:10.581195+0000","last_fullsized":"2023-12-11T19:32:10.581195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585052+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:32:10.585052+0000","last_peered":"2023-12-11T19:32:10.585052+0000","last_clean":"2023-12-11T19:32:10.585052+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:32:10.585052+0000","last_undegraded":"2023-12-11T19:32:10.585052+0000","last_fullsized":"2023-12-11T19:32:10.585052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581346+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:32:10.581346+0000","last_peered":"2023-12-11T19:32:10.581346+0000","last_clean":"2023-12-11T19:32:10.581346+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:32:10.581346+0000","last_undegraded":"2023-12-11T19:32:10.581346+0000","last_fullsized":"2023-12-11T19:32:10.581346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582290+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:32:10.582290+0000","last_peered":"2023-12-11T19:32:10.582290+0000","last_clean":"2023-12-11T19:32:10.582290+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:32:10.582290+0000","last_undegraded":"2023-12-11T19:32:10.582290+0000","last_fullsized":"2023-12-11T19:32:10.582290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.646420+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:32:44.646420+0000","last_peered":"2023-12-11T19:32:44.646420+0000","last_clean":"2023-12-11T19:32:44.646420+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:32:44.646420+0000","last_undegraded":"2023-12-11T19:32:44.646420+0000","last_fullsized":"2023-12-11T19:32:44.646420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583378+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:32:10.583378+0000","last_peered":"2023-12-11T19:32:10.583378+0000","last_clean":"2023-12-11T19:32:10.583378+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:32:10.583378+0000","last_undegraded":"2023-12-11T19:32:10.583378+0000","last_fullsized":"2023-12-11T19:32:10.583378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578707+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:32:10.578707+0000","last_peered":"2023-12-11T19:32:10.578707+0000","last_clean":"2023-12-11T19:32:10.578707+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:32:10.578707+0000","last_undegraded":"2023-12-11T19:32:10.578707+0000","last_fullsized":"2023-12-11T19:32:10.578707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579175+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:32:10.579175+0000","last_peered":"2023-12-11T19:32:10.579175+0000","last_clean":"2023-12-11T19:32:10.579175+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:32:10.579175+0000","last_undegraded":"2023-12-11T19:32:10.579175+0000","last_fullsized":"2023-12-11T19:32:10.579175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581870+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:32:10.581870+0000","last_peered":"2023-12-11T19:32:10.581870+0000","last_clean":"2023-12-11T19:32:10.581870+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:32:10.581870+0000","last_undegraded":"2023-12-11T19:32:10.581870+0000","last_fullsized":"2023-12-11T19:32:10.581870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582888+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:32:10.582888+0000","last_peered":"2023-12-11T19:32:10.582888+0000","last_clean":"2023-12-11T19:32:10.582888+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:32:10.582888+0000","last_undegraded":"2023-12-11T19:32:10.582888+0000","last_fullsized":"2023-12-11T19:32:10.582888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":150,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.567355+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:32:44.567355+0000","last_peered":"2023-12-11T19:32:44.567355+0000","last_clean":"2023-12-11T19:32:44.567355+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:32:44.567355+0000","last_undegraded":"2023-12-11T19:32:44.567355+0000","last_fullsized":"2023-12-11T19:32:44.567355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60009,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":111,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584887+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:32:10.584887+0000","last_peered":"2023-12-11T19:32:10.584887+0000","last_clean":"2023-12-11T19:32:10.584887+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:32:10.584887+0000","last_undegraded":"2023-12-11T19:32:10.584887+0000","last_fullsized":"2023-12-11T19:32:10.584887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.587382+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:32:10.587382+0000","last_peered":"2023-12-11T19:32:10.587382+0000","last_clean":"2023-12-11T19:32:10.587382+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:32:10.587382+0000","last_undegraded":"2023-12-11T19:32:10.587382+0000","last_fullsized":"2023-12-11T19:32:10.587382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581911+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:32:10.581911+0000","last_peered":"2023-12-11T19:32:10.581911+0000","last_clean":"2023-12-11T19:32:10.581911+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:32:10.581911+0000","last_undegraded":"2023-12-11T19:32:10.581911+0000","last_fullsized":"2023-12-11T19:32:10.581911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581282+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:32:10.581282+0000","last_peered":"2023-12-11T19:32:10.581282+0000","last_clean":"2023-12-11T19:32:10.581282+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:32:10.581282+0000","last_undegraded":"2023-12-11T19:32:10.581282+0000","last_fullsized":"2023-12-11T19:32:10.581282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578263+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:32:10.578263+0000","last_peered":"2023-12-11T19:32:10.578263+0000","last_clean":"2023-12-11T19:32:10.578263+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:32:10.578263+0000","last_undegraded":"2023-12-11T19:32:10.578263+0000","last_fullsized":"2023-12-11T19:32:10.578263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:41.633828+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:32:41.633828+0000","last_peered":"2023-12-11T19:32:41.633828+0000","last_clean":"2023-12-11T19:32:41.633828+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:32:41.633828+0000","last_undegraded":"2023-12-11T19:32:41.633828+0000","last_fullsized":"2023-12-11T19:32:41.633828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.577975+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:32:10.577975+0000","last_peered":"2023-12-11T19:32:10.577975+0000","last_clean":"2023-12-11T19:32:10.577975+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:32:10.577975+0000","last_undegraded":"2023-12-11T19:32:10.577975+0000","last_fullsized":"2023-12-11T19:32:10.577975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583186+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:32:10.583186+0000","last_peered":"2023-12-11T19:32:10.583186+0000","last_clean":"2023-12-11T19:32:10.583186+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:32:10.583186+0000","last_undegraded":"2023-12-11T19:32:10.583186+0000","last_fullsized":"2023-12-11T19:32:10.583186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581116+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:32:10.581116+0000","last_peered":"2023-12-11T19:32:10.581116+0000","last_clean":"2023-12-11T19:32:10.581116+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:32:10.581116+0000","last_undegraded":"2023-12-11T19:32:10.581116+0000","last_fullsized":"2023-12-11T19:32:10.581116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585021+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:32:10.585021+0000","last_peered":"2023-12-11T19:32:10.585021+0000","last_clean":"2023-12-11T19:32:10.585021+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:32:10.585021+0000","last_undegraded":"2023-12-11T19:32:10.585021+0000","last_fullsized":"2023-12-11T19:32:10.585021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583296+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:32:10.583296+0000","last_peered":"2023-12-11T19:32:10.583296+0000","last_clean":"2023-12-11T19:32:10.583296+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:32:10.583296+0000","last_undegraded":"2023-12-11T19:32:10.583296+0000","last_fullsized":"2023-12-11T19:32:10.583296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583392+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:32:10.583392+0000","last_peered":"2023-12-11T19:32:10.583392+0000","last_clean":"2023-12-11T19:32:10.583392+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:32:10.583392+0000","last_undegraded":"2023-12-11T19:32:10.583392+0000","last_fullsized":"2023-12-11T19:32:10.583392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585711+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:32:10.585711+0000","last_peered":"2023-12-11T19:32:10.585711+0000","last_clean":"2023-12-11T19:32:10.585711+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:32:10.585711+0000","last_undegraded":"2023-12-11T19:32:10.585711+0000","last_fullsized":"2023-12-11T19:32:10.585711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:42.624744+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:32:42.624744+0000","last_peered":"2023-12-11T19:32:42.624744+0000","last_clean":"2023-12-11T19:32:42.624744+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:32:42.624744+0000","last_undegraded":"2023-12-11T19:32:42.624744+0000","last_fullsized":"2023-12-11T19:32:42.624744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585788+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:32:10.585788+0000","last_peered":"2023-12-11T19:32:10.585788+0000","last_clean":"2023-12-11T19:32:10.585788+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:32:10.585788+0000","last_undegraded":"2023-12-11T19:32:10.585788+0000","last_fullsized":"2023-12-11T19:32:10.585788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.594663+0000","last_change":"2023-12-11T19:32:11.594139+0000","last_active":"2023-12-11T19:32:11.594663+0000","last_peered":"2023-12-11T19:32:11.594663+0000","last_clean":"2023-12-11T19:32:11.594663+0000","last_became_active":"2023-12-11T19:32:11.593876+0000","last_became_peered":"2023-12-11T19:32:11.593876+0000","last_unstale":"2023-12-11T19:32:11.594663+0000","last_undegraded":"2023-12-11T19:32:11.594663+0000","last_fullsized":"2023-12-11T19:32:11.594663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:22:15.243111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:39.628914+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:32:39.628914+0000","last_peered":"2023-12-11T19:32:39.628914+0000","last_clean":"2023-12-11T19:32:39.628914+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:32:39.628914+0000","last_undegraded":"2023-12-11T19:32:39.628914+0000","last_fullsized":"2023-12-11T19:32:39.628914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585233+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:32:10.585233+0000","last_peered":"2023-12-11T19:32:10.585233+0000","last_clean":"2023-12-11T19:32:10.585233+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:32:10.585233+0000","last_undegraded":"2023-12-11T19:32:10.585233+0000","last_fullsized":"2023-12-11T19:32:10.585233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582019+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:32:10.582019+0000","last_peered":"2023-12-11T19:32:10.582019+0000","last_clean":"2023-12-11T19:32:10.582019+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:32:10.582019+0000","last_undegraded":"2023-12-11T19:32:10.582019+0000","last_fullsized":"2023-12-11T19:32:10.582019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578778+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:32:10.578778+0000","last_peered":"2023-12-11T19:32:10.578778+0000","last_clean":"2023-12-11T19:32:10.578778+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:32:10.578778+0000","last_undegraded":"2023-12-11T19:32:10.578778+0000","last_fullsized":"2023-12-11T19:32:10.578778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.590972+0000","last_change":"2023-12-11T19:32:11.590972+0000","last_active":"2023-12-11T19:32:11.590972+0000","last_peered":"2023-12-11T19:32:11.590972+0000","last_clean":"2023-12-11T19:32:11.590972+0000","last_became_active":"2023-12-11T19:32:11.590564+0000","last_became_peered":"2023-12-11T19:32:11.590564+0000","last_unstale":"2023-12-11T19:32:11.590972+0000","last_undegraded":"2023-12-11T19:32:11.590972+0000","last_fullsized":"2023-12-11T19:32:11.590972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:48:35.323725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578698+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:32:10.578698+0000","last_peered":"2023-12-11T19:32:10.578698+0000","last_clean":"2023-12-11T19:32:10.578698+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:32:10.578698+0000","last_undegraded":"2023-12-11T19:32:10.578698+0000","last_fullsized":"2023-12-11T19:32:10.578698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584892+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:32:10.584892+0000","last_peered":"2023-12-11T19:32:10.584892+0000","last_clean":"2023-12-11T19:32:10.584892+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:32:10.584892+0000","last_undegraded":"2023-12-11T19:32:10.584892+0000","last_fullsized":"2023-12-11T19:32:10.584892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581417+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:32:10.581417+0000","last_peered":"2023-12-11T19:32:10.581417+0000","last_clean":"2023-12-11T19:32:10.581417+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:32:10.581417+0000","last_undegraded":"2023-12-11T19:32:10.581417+0000","last_fullsized":"2023-12-11T19:32:10.581417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599025+0000","last_change":"2023-12-11T19:32:11.599025+0000","last_active":"2023-12-11T19:32:11.599025+0000","last_peered":"2023-12-11T19:32:11.599025+0000","last_clean":"2023-12-11T19:32:11.599025+0000","last_became_active":"2023-12-11T19:32:11.587530+0000","last_became_peered":"2023-12-11T19:32:11.587530+0000","last_unstale":"2023-12-11T19:32:11.599025+0000","last_undegraded":"2023-12-11T19:32:11.599025+0000","last_fullsized":"2023-12-11T19:32:11.599025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:08:05.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586941+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:32:10.586941+0000","last_peered":"2023-12-11T19:32:10.586941+0000","last_clean":"2023-12-11T19:32:10.586941+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:32:10.586941+0000","last_undegraded":"2023-12-11T19:32:10.586941+0000","last_fullsized":"2023-12-11T19:32:10.586941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581975+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:32:10.581975+0000","last_peered":"2023-12-11T19:32:10.581975+0000","last_clean":"2023-12-11T19:32:10.581975+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:32:10.581975+0000","last_undegraded":"2023-12-11T19:32:10.581975+0000","last_fullsized":"2023-12-11T19:32:10.581975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578880+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:32:10.578880+0000","last_peered":"2023-12-11T19:32:10.578880+0000","last_clean":"2023-12-11T19:32:10.578880+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:32:10.578880+0000","last_undegraded":"2023-12-11T19:32:10.578880+0000","last_fullsized":"2023-12-11T19:32:10.578880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581192+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:32:10.581192+0000","last_peered":"2023-12-11T19:32:10.581192+0000","last_clean":"2023-12-11T19:32:10.581192+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:32:10.581192+0000","last_undegraded":"2023-12-11T19:32:10.581192+0000","last_fullsized":"2023-12-11T19:32:10.581192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582869+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:32:10.582869+0000","last_peered":"2023-12-11T19:32:10.582869+0000","last_clean":"2023-12-11T19:32:10.582869+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:32:10.582869+0000","last_undegraded":"2023-12-11T19:32:10.582869+0000","last_fullsized":"2023-12-11T19:32:10.582869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585191+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:32:10.585191+0000","last_peered":"2023-12-11T19:32:10.585191+0000","last_clean":"2023-12-11T19:32:10.585191+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:32:10.585191+0000","last_undegraded":"2023-12-11T19:32:10.585191+0000","last_fullsized":"2023-12-11T19:32:10.585191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581598+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:32:10.581598+0000","last_peered":"2023-12-11T19:32:10.581598+0000","last_clean":"2023-12-11T19:32:10.581598+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:32:10.581598+0000","last_undegraded":"2023-12-11T19:32:10.581598+0000","last_fullsized":"2023-12-11T19:32:10.581598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586254+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:32:10.586254+0000","last_peered":"2023-12-11T19:32:10.586254+0000","last_clean":"2023-12-11T19:32:10.586254+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:32:10.586254+0000","last_undegraded":"2023-12-11T19:32:10.586254+0000","last_fullsized":"2023-12-11T19:32:10.586254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586164+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:32:10.586164+0000","last_peered":"2023-12-11T19:32:10.586164+0000","last_clean":"2023-12-11T19:32:10.586164+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:32:10.586164+0000","last_undegraded":"2023-12-11T19:32:10.586164+0000","last_fullsized":"2023-12-11T19:32:10.586164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599949+0000","last_change":"2023-12-11T19:32:11.599116+0000","last_active":"2023-12-11T19:32:11.599949+0000","last_peered":"2023-12-11T19:32:11.599949+0000","last_clean":"2023-12-11T19:32:11.599949+0000","last_became_active":"2023-12-11T19:32:11.592106+0000","last_became_peered":"2023-12-11T19:32:11.592106+0000","last_unstale":"2023-12-11T19:32:11.599949+0000","last_undegraded":"2023-12-11T19:32:11.599949+0000","last_fullsized":"2023-12-11T19:32:11.599949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:23.285224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578288+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:32:10.578288+0000","last_peered":"2023-12-11T19:32:10.578288+0000","last_clean":"2023-12-11T19:32:10.578288+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:32:10.578288+0000","last_undegraded":"2023-12-11T19:32:10.578288+0000","last_fullsized":"2023-12-11T19:32:10.578288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.649432+0000","last_change":"2023-12-11T19:32:11.603249+0000","last_active":"2023-12-11T19:32:44.649432+0000","last_peered":"2023-12-11T19:32:44.649432+0000","last_clean":"2023-12-11T19:32:44.649432+0000","last_became_active":"2023-12-11T19:32:11.587129+0000","last_became_peered":"2023-12-11T19:32:11.587129+0000","last_unstale":"2023-12-11T19:32:44.649432+0000","last_undegraded":"2023-12-11T19:32:44.649432+0000","last_fullsized":"2023-12-11T19:32:44.649432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:28:44.762360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578593+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:32:10.578593+0000","last_peered":"2023-12-11T19:32:10.578593+0000","last_clean":"2023-12-11T19:32:10.578593+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:32:10.578593+0000","last_undegraded":"2023-12-11T19:32:10.578593+0000","last_fullsized":"2023-12-11T19:32:10.578593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581433+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:32:10.581433+0000","last_peered":"2023-12-11T19:32:10.581433+0000","last_clean":"2023-12-11T19:32:10.581433+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:32:10.581433+0000","last_undegraded":"2023-12-11T19:32:10.581433+0000","last_fullsized":"2023-12-11T19:32:10.581433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.596576+0000","last_change":"2023-12-11T19:32:11.596576+0000","last_active":"2023-12-11T19:32:11.596576+0000","last_peered":"2023-12-11T19:32:11.596576+0000","last_clean":"2023-12-11T19:32:11.596576+0000","last_became_active":"2023-12-11T19:32:11.592679+0000","last_became_peered":"2023-12-11T19:32:11.592679+0000","last_unstale":"2023-12-11T19:32:11.596576+0000","last_undegraded":"2023-12-11T19:32:11.596576+0000","last_fullsized":"2023-12-11T19:32:11.596576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:26:20.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581645+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:32:10.581645+0000","last_peered":"2023-12-11T19:32:10.581645+0000","last_clean":"2023-12-11T19:32:10.581645+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:32:10.581645+0000","last_undegraded":"2023-12-11T19:32:10.581645+0000","last_fullsized":"2023-12-11T19:32:10.581645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578665+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:32:10.578665+0000","last_peered":"2023-12-11T19:32:10.578665+0000","last_clean":"2023-12-11T19:32:10.578665+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:32:10.578665+0000","last_undegraded":"2023-12-11T19:32:10.578665+0000","last_fullsized":"2023-12-11T19:32:10.578665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581698+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:32:10.581698+0000","last_peered":"2023-12-11T19:32:10.581698+0000","last_clean":"2023-12-11T19:32:10.581698+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:32:10.581698+0000","last_undegraded":"2023-12-11T19:32:10.581698+0000","last_fullsized":"2023-12-11T19:32:10.581698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586080+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:32:10.586080+0000","last_peered":"2023-12-11T19:32:10.586080+0000","last_clean":"2023-12-11T19:32:10.586080+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:32:10.586080+0000","last_undegraded":"2023-12-11T19:32:10.586080+0000","last_fullsized":"2023-12-11T19:32:10.586080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586165+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:32:10.586165+0000","last_peered":"2023-12-11T19:32:10.586165+0000","last_clean":"2023-12-11T19:32:10.586165+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:32:10.586165+0000","last_undegraded":"2023-12-11T19:32:10.586165+0000","last_fullsized":"2023-12-11T19:32:10.586165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.586739+0000","last_change":"2023-12-11T19:32:11.586739+0000","last_active":"2023-12-11T19:32:11.586739+0000","last_peered":"2023-12-11T19:32:11.586739+0000","last_clean":"2023-12-11T19:32:11.586739+0000","last_became_active":"2023-12-11T19:32:11.586457+0000","last_became_peered":"2023-12-11T19:32:11.586457+0000","last_unstale":"2023-12-11T19:32:11.586739+0000","last_undegraded":"2023-12-11T19:32:11.586739+0000","last_fullsized":"2023-12-11T19:32:11.586739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:50.986234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582884+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:32:10.582884+0000","last_peered":"2023-12-11T19:32:10.582884+0000","last_clean":"2023-12-11T19:32:10.582884+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:32:10.582884+0000","last_undegraded":"2023-12-11T19:32:10.582884+0000","last_fullsized":"2023-12-11T19:32:10.582884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583047+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:32:10.583047+0000","last_peered":"2023-12-11T19:32:10.583047+0000","last_clean":"2023-12-11T19:32:10.583047+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:32:10.583047+0000","last_undegraded":"2023-12-11T19:32:10.583047+0000","last_fullsized":"2023-12-11T19:32:10.583047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586827+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:32:10.586827+0000","last_peered":"2023-12-11T19:32:10.586827+0000","last_clean":"2023-12-11T19:32:10.586827+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:32:10.586827+0000","last_undegraded":"2023-12-11T19:32:10.586827+0000","last_fullsized":"2023-12-11T19:32:10.586827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586532+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:32:10.586532+0000","last_peered":"2023-12-11T19:32:10.586532+0000","last_clean":"2023-12-11T19:32:10.586532+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:32:10.586532+0000","last_undegraded":"2023-12-11T19:32:10.586532+0000","last_fullsized":"2023-12-11T19:32:10.586532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581431+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:32:10.581431+0000","last_peered":"2023-12-11T19:32:10.581431+0000","last_clean":"2023-12-11T19:32:10.581431+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:32:10.581431+0000","last_undegraded":"2023-12-11T19:32:10.581431+0000","last_fullsized":"2023-12-11T19:32:10.581431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585160+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:32:10.585160+0000","last_peered":"2023-12-11T19:32:10.585160+0000","last_clean":"2023-12-11T19:32:10.585160+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:32:10.585160+0000","last_undegraded":"2023-12-11T19:32:10.585160+0000","last_fullsized":"2023-12-11T19:32:10.585160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582513+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:32:10.582513+0000","last_peered":"2023-12-11T19:32:10.582513+0000","last_clean":"2023-12-11T19:32:10.582513+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:32:10.582513+0000","last_undegraded":"2023-12-11T19:32:10.582513+0000","last_fullsized":"2023-12-11T19:32:10.582513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581557+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:32:10.581557+0000","last_peered":"2023-12-11T19:32:10.581557+0000","last_clean":"2023-12-11T19:32:10.581557+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:32:10.581557+0000","last_undegraded":"2023-12-11T19:32:10.581557+0000","last_fullsized":"2023-12-11T19:32:10.581557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582438+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:32:10.582438+0000","last_peered":"2023-12-11T19:32:10.582438+0000","last_clean":"2023-12-11T19:32:10.582438+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:32:10.582438+0000","last_undegraded":"2023-12-11T19:32:10.582438+0000","last_fullsized":"2023-12-11T19:32:10.582438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67209,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":483,"num_read_kb":320,"num_write":230,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2181,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":201627,"data_compressed":5496,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":459769,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.72999999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.623}]},{"osd":2,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.60399999999999998}]},{"osd":6,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80900000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":9,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.004},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.96899999999999997}]},{"osd":10,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.0489999999999999}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1111202,"data_compressed":43838,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.503},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54300000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82499999999999996,"5min":1.04,"15min":1.04},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.442,"15min":0.442},"max":{"1min":0.94699999999999995,"5min":1.032,"15min":1.032},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.496,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.61299999999999999}]},{"osd":4,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.69999999999999996,"15min":0.71299999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92300000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98499999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.54900000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.085,"15min":1.085},"last":0.98399999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.85299999999999998,"15min":0.83899999999999997},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0089999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":0.90500000000000003},"min":{"1min":0.66500000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0640000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92200000000000004,"15min":0.93799999999999994},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0609999999999999,"5min":1.643,"15min":1.643},"last":0.90800000000000003}]},{"osd":9,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90400000000000003,"15min":0.90800000000000003},"min":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.034,"5min":1.321,"15min":1.321},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0309999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.747}]},{"osd":11,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0009999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":459822,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.49399999999999999}]},{"osd":1,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.83899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0609999999999999,"5min":1.349,"15min":1.349},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.44},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74299999999999999,"15min":0.753},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.62}]},{"osd":4,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.90800000000000003,"15min":0.92500000000000004},"min":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":1.278,"15min":1.278},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.90100000000000002,"15min":0.91500000000000004},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.115,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0429999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90500000000000003,"15min":0.93200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.389,"15min":1.389},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0960000000000001,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.871},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.97299999999999998,"15min":0.98599999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2250000000000001,"5min":1.274,"15min":1.274},"last":0.84399999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0009999999999999,"15min":1.018},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1850000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.0089999999999999,"15min":1.0249999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.097,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.98599999999999999,"15min":1.0069999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.98599999999999999,"15min":0.998},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.169,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":460077,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.751},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":1.097,"15min":1.097},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.55700000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1479999999999999,"5min":1.351,"15min":1.351},"last":0.996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.81200000000000006,"15min":0.80900000000000005},"min":{"1min":0.61399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.0740000000000001,"5min":2.21,"15min":2.21},"last":0.80000000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.622},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.292,"15min":1.292},"last":0.77000000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.2350000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.58199999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1140000000000001,"5min":2.173,"15min":2.173},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.083,"5min":1.177,"15min":1.177},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.84899999999999998,"15min":0.82099999999999995},"min":{"1min":0.74199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2869999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1459999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.84899999999999998}]},{"osd":7,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.71699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.304,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.74099999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.95199999999999996}]},{"osd":9,"last update":"Mon Dec 11 19:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.92100000000000004,"15min":0.91000000000000003},"min":{"1min":0.71599999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.4470000000000001,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.751,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.083}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.79200000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.405,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.056,"5min":0.98299999999999998,"15min":0.97099999999999997},"min":{"1min":0.77600000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.343,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.1180000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1052657,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.88500000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80600000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.94999999999999996}]},{"osd":3,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.78300000000000003,"15min":0.81499999999999995},"min":{"1min":0.51000000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79700000000000004,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.72299999999999998,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.92600000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65100000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75700000000000001,"15min":0.78400000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94699999999999995,"5min":1.238,"15min":1.238},"last":0.61199999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98599999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85699999999999998,"15min":0.86299999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1000000000000001,"5min":1.28,"15min":1.28},"last":0.87}]},{"osd":6,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0800000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77700000000000002,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.052,"5min":1.359,"15min":1.359},"last":0.63600000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97999999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.85999999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.248,"15min":1.248},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.89500000000000002,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95799999999999996,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0920000000000001,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":459422,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.31,"15min":0.31},"max":{"1min":0.68400000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.74399999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92500000000000004,"5min":1.284,"15min":1.284},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95599999999999996,"5min":1.228,"15min":1.228},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.016,"15min":1.016},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.119,"5min":1.157,"15min":1.157},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82399999999999995,"15min":0.81200000000000006},"min":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88600000000000001,"15min":0.92200000000000004},"min":{"1min":0.65400000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0349999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.81799999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.089,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.879,"15min":0.86799999999999999},"min":{"1min":0.629,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.181,"5min":1.234,"15min":1.234},"last":0.95699999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.85099999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.92200000000000004},"min":{"1min":0.67100000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.86699999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1859999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.88700000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.84899999999999998,"15min":0.83299999999999996},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.012,"5min":0.98099999999999998,"15min":0.97599999999999998},"min":{"1min":0.84899999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.081}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521235,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.55700000000000005},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69099999999999995,"15min":0.58199999999999996},"min":{"1min":0.58099999999999996,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":1.05,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.081,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.65600000000000003},"min":{"1min":0.50900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1779999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.81899999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.50900000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91100000000000003,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.629}]},{"osd":4,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.437},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.60399999999999998,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Dec 11 19:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.70799999999999996,"15min":0.66800000000000004},"min":{"1min":0.56100000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.135,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.77000000000000002,"15min":0.86699999999999999},"min":{"1min":0.54700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92800000000000005,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.63,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.05,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.69699999999999995,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64600000000000002,"15min":0.61599999999999999},"min":{"1min":0.57199999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.501}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2110000000000001,"5min":1.52,"15min":1.52},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.091,"5min":0.88500000000000001,"15min":0.83799999999999997},"min":{"1min":0.67100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.95999999999999996}]},{"osd":9,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.88400000000000001,"15min":0.86399999999999999},"min":{"1min":0.79000000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.89000000000000001,"15min":0.875},"min":{"1min":0.752,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.84499999999999997}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.83599999999999997,"15min":0.82799999999999996},"min":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0840000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93300000000000005},"min":{"1min":0.76400000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.196,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":460755,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82799999999999996,"15min":0.67800000000000005},"min":{"1min":0.52300000000000002,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":1.1890000000000001,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":1.0449999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.71599999999999997}]},{"osd":2,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71499999999999997,"15min":0.63500000000000001},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1120000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.439,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94299999999999995,"5min":1.403,"15min":1.403},"last":0.878}]},{"osd":3,"last update":"Mon Dec 11 19:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.46100000000000002},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77200000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.749},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":1.004,"15min":1.004},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Mon Dec 11 19:32:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89800000000000002,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.95099999999999996,"5min":1.819,"15min":1.819},"last":1.0189999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:31:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.079},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.004}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.71299999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.103},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.89600000000000002},"min":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.98999999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.98499999999999999,"15min":1.014},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3049999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.93100000000000005,"15min":0.96299999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96899999999999997,"5min":1.202,"15min":1.202},"last":1.04}]},{"osd":10,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.143},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.97499999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":6.9299999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":518766,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.502},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.44400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69099999999999995,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.98999999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82799999999999996}]},{"osd":3,"last update":"Mon Dec 11 19:32:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.0249999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.58999999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.86399999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59799999999999998,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93400000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.627}]},{"osd":6,"last update":"Mon Dec 11 19:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.90100000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.84099999999999997,"15min":0.82699999999999996},"min":{"1min":0.64500000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.70599999999999996,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.91900000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.90400000000000003,"15min":0.89200000000000002},"min":{"1min":0.73399999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.996,"5min":0.90000000000000002,"15min":0.88400000000000001},"min":{"1min":0.78400000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0129999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":458799,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.40500000000000003},"min":{"1min":0.38900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.501,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65800000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.68799999999999994,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.63800000000000001},"min":{"1min":0.57099999999999995,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.64300000000000002},"min":{"1min":0.503,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Dec 11 19:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.054,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.75600000000000001,"15min":0.69999999999999996},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.105,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.746}]},{"osd":5,"last update":"Mon Dec 11 19:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.76100000000000001,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:32:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.82599999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.621},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.82899999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.96099999999999997,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0089999999999999,"15min":1.02},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.347,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.627}]},{"osd":10,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.98699999999999999,"15min":0.98399999999999999},"min":{"1min":0.79600000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.056,"15min":1.0669999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":460677,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.46400000000000002},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.626},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.48299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67900000000000005,"15min":0.65200000000000002},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94999999999999996,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Dec 11 19:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73699999999999999,"15min":0.69699999999999995},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.361,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72699999999999998,"15min":0.71699999999999997},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.76600000000000001}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97499999999999998,"5min":1.075,"15min":1.075},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.54600000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.83299999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:32:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72299999999999998,"15min":0.70699999999999996},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.95999999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.67600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.98899999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88700000000000001,"15min":0.88300000000000001},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4399999999999999,"5min":1.494,"15min":1.494},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.874}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90200000000000002,"15min":0.89900000000000002},"min":{"1min":0.68799999999999994,"5min":0.628,"15min":0.628},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.018},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.95599999999999996,"15min":0.95899999999999996},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5029999999999999,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":1052277,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.63100000000000001,"15min":0.65900000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":1.177,"15min":1.177},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67700000000000005,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0409999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85899999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.502,"15min":0.46600000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.754},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.78200000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.141,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71499999999999997,"15min":0.74099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76300000000000001,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.746},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.248,"5min":1.597,"15min":1.597},"last":0.83899999999999997}]},{"osd":7,"last update":"Mon Dec 11 19:32:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76400000000000001,"15min":0.745},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1659999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.999},{"interface":"front","average":{"1min":0.879,"5min":0.81699999999999995,"15min":0.80100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.02}]},{"osd":8,"last update":"Mon Dec 11 19:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.214,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0740000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.61499999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1819999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.88500000000000001,"15min":0.88900000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1759999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.05}]},{"osd":10,"last update":"Mon Dec 11 19:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90200000000000002,"15min":0.90500000000000003},"min":{"1min":0.66900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.256,"5min":1.548,"15min":1.548},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92900000000000005,"15min":0.92600000000000005},"min":{"1min":0.63700000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.1220000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62931,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60189,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62134,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:32:47.294 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph pg dump --format=json 2023-12-11T19:32:48.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:47 smithi167 ceph-mon[102750]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:32:48.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:47 smithi167 ceph-mon[102750]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:48.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:47 smithi066 ceph-mon[97920]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:32:48.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:47 smithi066 ceph-mon[97920]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:48.095 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:48.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:47 smithi070 ceph-mon[104044]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:32:48.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:47 smithi070 ceph-mon[104044]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:49.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.634+0000 7fa3f56de700 1 -- 172.21.15.66:0/1064623682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0069930 msgr2=0x7fa3f010e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.634+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1064623682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0069930 0x7fa3f010e420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800b0d0 tx=0x7fa3e800b3e0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 -- 172.21.15.66:0/1064623682 shutdown_connections 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1064623682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0107620 0x7fa3f0107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1064623682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0069930 0x7fa3f010e420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1064623682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0068f70 0x7fa3f00693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 -- 172.21.15.66:0/1064623682 >> 172.21.15.66:0/1064623682 conn(0x7fa3f0076570 msgr2=0x7fa3f0076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 -- 172.21.15.66:0/1064623682 shutdown_connections 2023-12-11T19:32:49.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.635+0000 7fa3f56de700 1 -- 172.21.15.66:0/1064623682 wait complete. 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.636+0000 7fa3f56de700 1 Processor -- start 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.636+0000 7fa3f56de700 1 -- start start 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.636+0000 7fa3f56de700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 0x7fa3f019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.636+0000 7fa3f56de700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0069930 0x7fa3f019ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3f56de700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0107620 0x7fa3f0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3f56de700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f019a260 con 0x7fa3f0068f70 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3f56de700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f019a3d0 con 0x7fa3f0107620 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3f56de700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f019a590 con 0x7fa3f0069930 2023-12-11T19:32:49.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 0x7fa3f019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 0x7fa3f019fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49542/0 (socket says 172.21.15.66:49542) 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 -- 172.21.15.66:0/1547134546 learned_addr learned my addr 172.21.15.66:0/1547134546 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3ee7fc700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0069930 0x7fa3f019ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 -- 172.21.15.66:0/1547134546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0069930 msgr2=0x7fa3f019ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0069930 0x7fa3f019ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 -- 172.21.15.66:0/1547134546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0107620 msgr2=0x7fa3f0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0107620 0x7fa3f0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3eeffd700 1 -- 172.21.15.66:0/1547134546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8009d00 con 0x7fa3f0068f70 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.637+0000 7fa3ee7fc700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0069930 0x7fa3f019ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:49.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.638+0000 7fa3eeffd700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 0x7fa3f019fa00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d950 tx=0x7fa3e000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:49.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.638+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0009b40 con 0x7fa3f0068f70 2023-12-11T19:32:49.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.638+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3e0009ca0 con 0x7fa3f0068f70 2023-12-11T19:32:49.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.638+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f019a870 con 0x7fa3f0068f70 2023-12-11T19:32:49.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.638+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000f620 con 0x7fa3f0068f70 2023-12-11T19:32:49.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.638+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f01a69d0 con 0x7fa3f0068f70 2023-12-11T19:32:49.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.639+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f0102cc0 con 0x7fa3f0068f70 2023-12-11T19:32:49.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.641+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa3e0010460 con 0x7fa3f0068f70 2023-12-11T19:32:49.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.642+0000 7fa3d7fff700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3d80a1590 0x7fa3d80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:49.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.642+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3e00cd100 con 0x7fa3f0068f70 2023-12-11T19:32:49.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.643+0000 7fa3ee7fc700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3d80a1590 0x7fa3d80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:49.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.643+0000 7fa3ee7fc700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3d80a1590 0x7fa3d80a3a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005fd0 tx=0x7fa3e4005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:49.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.646+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa3e00965c0 con 0x7fa3f0068f70 2023-12-11T19:32:49.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.852+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3f01113b0 con 0x7fa3d80a1590 2023-12-11T19:32:49.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.863+0000 7fa3d7fff700 1 -- 172.21.15.66:0/1547134546 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339903 (secure 0 0 0) 0x7fa3f01113b0 con 0x7fa3d80a1590 2023-12-11T19:32:49.869 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:49.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.867+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3d80a1590 msgr2=0x7fa3d80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:49.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.867+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3d80a1590 0x7fa3d80a3a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005fd0 tx=0x7fa3e4005ee0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.867+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 msgr2=0x7fa3f019fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:49.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.867+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 0x7fa3f019fa00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d950 tx=0x7fa3e000dd10 comp rx=0 tx=0).stop 2023-12-11T19:32:49.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 shutdown_connections 2023-12-11T19:32:49.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa3d80a1590 0x7fa3d80a3a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f0107620 0x7fa3f0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0068f70 0x7fa3f019fa00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 --2- 172.21.15.66:0/1547134546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0069930 0x7fa3f019ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 >> 172.21.15.66:0/1547134546 conn(0x7fa3f0076570 msgr2=0x7fa3f010c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 shutdown_connections 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:49.868+0000 7fa3f56de700 1 -- 172.21.15.66:0/1547134546 wait complete. 2023-12-11T19:32:49.873 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:32:50.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:49 smithi167 ceph-mon[102750]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:50.060 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:49 smithi066 ceph-mon[97920]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:50.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:49 smithi070 ceph-mon[104044]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:50.403 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2023-12-11T19:32:48.299013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657823,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":619,"num_read_kb":434,"num_write":413,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1183040,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336032,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605792,"statfs":{"total":1151940427776,"available":1151596331008,"internally_reserved":0,"allocated":12066816,"data_stored":7475458,"data_compressed":183741,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003126"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583470+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:32:10.583470+0000","last_peered":"2023-12-11T19:32:10.583470+0000","last_clean":"2023-12-11T19:32:10.583470+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:32:10.583470+0000","last_undegraded":"2023-12-11T19:32:10.583470+0000","last_fullsized":"2023-12-11T19:32:10.583470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579009+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:32:10.579009+0000","last_peered":"2023-12-11T19:32:10.579009+0000","last_clean":"2023-12-11T19:32:10.579009+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:32:10.579009+0000","last_undegraded":"2023-12-11T19:32:10.579009+0000","last_fullsized":"2023-12-11T19:32:10.579009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579128+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:32:10.579128+0000","last_peered":"2023-12-11T19:32:10.579128+0000","last_clean":"2023-12-11T19:32:10.579128+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:32:10.579128+0000","last_undegraded":"2023-12-11T19:32:10.579128+0000","last_fullsized":"2023-12-11T19:32:10.579128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582246+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:32:10.582246+0000","last_peered":"2023-12-11T19:32:10.582246+0000","last_clean":"2023-12-11T19:32:10.582246+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:32:10.582246+0000","last_undegraded":"2023-12-11T19:32:10.582246+0000","last_fullsized":"2023-12-11T19:32:10.582246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584906+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:32:10.584906+0000","last_peered":"2023-12-11T19:32:10.584906+0000","last_clean":"2023-12-11T19:32:10.584906+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:32:10.584906+0000","last_undegraded":"2023-12-11T19:32:10.584906+0000","last_fullsized":"2023-12-11T19:32:10.584906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.585539+0000","last_change":"2023-12-11T19:32:11.585539+0000","last_active":"2023-12-11T19:32:11.585539+0000","last_peered":"2023-12-11T19:32:11.585539+0000","last_clean":"2023-12-11T19:32:11.585539+0000","last_became_active":"2023-12-11T19:32:11.585087+0000","last_became_peered":"2023-12-11T19:32:11.585087+0000","last_unstale":"2023-12-11T19:32:11.585539+0000","last_undegraded":"2023-12-11T19:32:11.585539+0000","last_fullsized":"2023-12-11T19:32:11.585539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:23.076166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578059+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:32:10.578059+0000","last_peered":"2023-12-11T19:32:10.578059+0000","last_clean":"2023-12-11T19:32:10.578059+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:32:10.578059+0000","last_undegraded":"2023-12-11T19:32:10.578059+0000","last_fullsized":"2023-12-11T19:32:10.578059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585017+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:32:10.585017+0000","last_peered":"2023-12-11T19:32:10.585017+0000","last_clean":"2023-12-11T19:32:10.585017+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:32:10.585017+0000","last_undegraded":"2023-12-11T19:32:10.585017+0000","last_fullsized":"2023-12-11T19:32:10.585017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583595+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:32:10.583595+0000","last_peered":"2023-12-11T19:32:10.583595+0000","last_clean":"2023-12-11T19:32:10.583595+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:32:10.583595+0000","last_undegraded":"2023-12-11T19:32:10.583595+0000","last_fullsized":"2023-12-11T19:32:10.583595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585811+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:32:10.585811+0000","last_peered":"2023-12-11T19:32:10.585811+0000","last_clean":"2023-12-11T19:32:10.585811+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:32:10.585811+0000","last_undegraded":"2023-12-11T19:32:10.585811+0000","last_fullsized":"2023-12-11T19:32:10.585811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578497+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:32:10.578497+0000","last_peered":"2023-12-11T19:32:10.578497+0000","last_clean":"2023-12-11T19:32:10.578497+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:32:10.578497+0000","last_undegraded":"2023-12-11T19:32:10.578497+0000","last_fullsized":"2023-12-11T19:32:10.578497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582199+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:32:10.582199+0000","last_peered":"2023-12-11T19:32:10.582199+0000","last_clean":"2023-12-11T19:32:10.582199+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:32:10.582199+0000","last_undegraded":"2023-12-11T19:32:10.582199+0000","last_fullsized":"2023-12-11T19:32:10.582199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582043+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:32:10.582043+0000","last_peered":"2023-12-11T19:32:10.582043+0000","last_clean":"2023-12-11T19:32:10.582043+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:32:10.582043+0000","last_undegraded":"2023-12-11T19:32:10.582043+0000","last_fullsized":"2023-12-11T19:32:10.582043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578129+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:32:10.578129+0000","last_peered":"2023-12-11T19:32:10.578129+0000","last_clean":"2023-12-11T19:32:10.578129+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:32:10.578129+0000","last_undegraded":"2023-12-11T19:32:10.578129+0000","last_fullsized":"2023-12-11T19:32:10.578129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.577960+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:32:10.577960+0000","last_peered":"2023-12-11T19:32:10.577960+0000","last_clean":"2023-12-11T19:32:10.577960+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:32:10.577960+0000","last_undegraded":"2023-12-11T19:32:10.577960+0000","last_fullsized":"2023-12-11T19:32:10.577960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581063+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:32:10.581063+0000","last_peered":"2023-12-11T19:32:10.581063+0000","last_clean":"2023-12-11T19:32:10.581063+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:32:10.581063+0000","last_undegraded":"2023-12-11T19:32:10.581063+0000","last_fullsized":"2023-12-11T19:32:10.581063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584127+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:32:10.584127+0000","last_peered":"2023-12-11T19:32:10.584127+0000","last_clean":"2023-12-11T19:32:10.584127+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:32:10.584127+0000","last_undegraded":"2023-12-11T19:32:10.584127+0000","last_fullsized":"2023-12-11T19:32:10.584127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584169+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:32:10.584169+0000","last_peered":"2023-12-11T19:32:10.584169+0000","last_clean":"2023-12-11T19:32:10.584169+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:32:10.584169+0000","last_undegraded":"2023-12-11T19:32:10.584169+0000","last_fullsized":"2023-12-11T19:32:10.584169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586383+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:32:10.586383+0000","last_peered":"2023-12-11T19:32:10.586383+0000","last_clean":"2023-12-11T19:32:10.586383+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:32:10.586383+0000","last_undegraded":"2023-12-11T19:32:10.586383+0000","last_fullsized":"2023-12-11T19:32:10.586383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:42.624082+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:32:42.624082+0000","last_peered":"2023-12-11T19:32:42.624082+0000","last_clean":"2023-12-11T19:32:42.624082+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:32:42.624082+0000","last_undegraded":"2023-12-11T19:32:42.624082+0000","last_fullsized":"2023-12-11T19:32:42.624082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586670+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:32:10.586670+0000","last_peered":"2023-12-11T19:32:10.586670+0000","last_clean":"2023-12-11T19:32:10.586670+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:32:10.586670+0000","last_undegraded":"2023-12-11T19:32:10.586670+0000","last_fullsized":"2023-12-11T19:32:10.586670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586855+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:32:10.586855+0000","last_peered":"2023-12-11T19:32:10.586855+0000","last_clean":"2023-12-11T19:32:10.586855+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:32:10.586855+0000","last_undegraded":"2023-12-11T19:32:10.586855+0000","last_fullsized":"2023-12-11T19:32:10.586855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579198+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:32:10.579198+0000","last_peered":"2023-12-11T19:32:10.579198+0000","last_clean":"2023-12-11T19:32:10.579198+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:32:10.579198+0000","last_undegraded":"2023-12-11T19:32:10.579198+0000","last_fullsized":"2023-12-11T19:32:10.579198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585009+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:32:10.585009+0000","last_peered":"2023-12-11T19:32:10.585009+0000","last_clean":"2023-12-11T19:32:10.585009+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:32:10.585009+0000","last_undegraded":"2023-12-11T19:32:10.585009+0000","last_fullsized":"2023-12-11T19:32:10.585009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582475+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:32:10.582475+0000","last_peered":"2023-12-11T19:32:10.582475+0000","last_clean":"2023-12-11T19:32:10.582475+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:32:10.582475+0000","last_undegraded":"2023-12-11T19:32:10.582475+0000","last_fullsized":"2023-12-11T19:32:10.582475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584594+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:32:10.584594+0000","last_peered":"2023-12-11T19:32:10.584594+0000","last_clean":"2023-12-11T19:32:10.584594+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:32:10.584594+0000","last_undegraded":"2023-12-11T19:32:10.584594+0000","last_fullsized":"2023-12-11T19:32:10.584594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581728+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:32:10.581728+0000","last_peered":"2023-12-11T19:32:10.581728+0000","last_clean":"2023-12-11T19:32:10.581728+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:32:10.581728+0000","last_undegraded":"2023-12-11T19:32:10.581728+0000","last_fullsized":"2023-12-11T19:32:10.581728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:42.643702+0000","last_change":"2023-12-11T19:32:11.601552+0000","last_active":"2023-12-11T19:32:42.643702+0000","last_peered":"2023-12-11T19:32:42.643702+0000","last_clean":"2023-12-11T19:32:42.643702+0000","last_became_active":"2023-12-11T19:32:11.593828+0000","last_became_peered":"2023-12-11T19:32:11.593828+0000","last_unstale":"2023-12-11T19:32:42.643702+0000","last_undegraded":"2023-12-11T19:32:42.643702+0000","last_fullsized":"2023-12-11T19:32:42.643702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:53.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583003+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:32:10.583003+0000","last_peered":"2023-12-11T19:32:10.583003+0000","last_clean":"2023-12-11T19:32:10.583003+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:32:10.583003+0000","last_undegraded":"2023-12-11T19:32:10.583003+0000","last_fullsized":"2023-12-11T19:32:10.583003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586092+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:32:10.586092+0000","last_peered":"2023-12-11T19:32:10.586092+0000","last_clean":"2023-12-11T19:32:10.586092+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:32:10.586092+0000","last_undegraded":"2023-12-11T19:32:10.586092+0000","last_fullsized":"2023-12-11T19:32:10.586092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582047+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:32:10.582047+0000","last_peered":"2023-12-11T19:32:10.582047+0000","last_clean":"2023-12-11T19:32:10.582047+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:32:10.582047+0000","last_undegraded":"2023-12-11T19:32:10.582047+0000","last_fullsized":"2023-12-11T19:32:10.582047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581195+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:32:10.581195+0000","last_peered":"2023-12-11T19:32:10.581195+0000","last_clean":"2023-12-11T19:32:10.581195+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:32:10.581195+0000","last_undegraded":"2023-12-11T19:32:10.581195+0000","last_fullsized":"2023-12-11T19:32:10.581195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585052+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:32:10.585052+0000","last_peered":"2023-12-11T19:32:10.585052+0000","last_clean":"2023-12-11T19:32:10.585052+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:32:10.585052+0000","last_undegraded":"2023-12-11T19:32:10.585052+0000","last_fullsized":"2023-12-11T19:32:10.585052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581346+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:32:10.581346+0000","last_peered":"2023-12-11T19:32:10.581346+0000","last_clean":"2023-12-11T19:32:10.581346+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:32:10.581346+0000","last_undegraded":"2023-12-11T19:32:10.581346+0000","last_fullsized":"2023-12-11T19:32:10.581346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582290+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:32:10.582290+0000","last_peered":"2023-12-11T19:32:10.582290+0000","last_clean":"2023-12-11T19:32:10.582290+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:32:10.582290+0000","last_undegraded":"2023-12-11T19:32:10.582290+0000","last_fullsized":"2023-12-11T19:32:10.582290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.646420+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:32:44.646420+0000","last_peered":"2023-12-11T19:32:44.646420+0000","last_clean":"2023-12-11T19:32:44.646420+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:32:44.646420+0000","last_undegraded":"2023-12-11T19:32:44.646420+0000","last_fullsized":"2023-12-11T19:32:44.646420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583378+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:32:10.583378+0000","last_peered":"2023-12-11T19:32:10.583378+0000","last_clean":"2023-12-11T19:32:10.583378+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:32:10.583378+0000","last_undegraded":"2023-12-11T19:32:10.583378+0000","last_fullsized":"2023-12-11T19:32:10.583378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578707+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:32:10.578707+0000","last_peered":"2023-12-11T19:32:10.578707+0000","last_clean":"2023-12-11T19:32:10.578707+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:32:10.578707+0000","last_undegraded":"2023-12-11T19:32:10.578707+0000","last_fullsized":"2023-12-11T19:32:10.578707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.579175+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:32:10.579175+0000","last_peered":"2023-12-11T19:32:10.579175+0000","last_clean":"2023-12-11T19:32:10.579175+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:32:10.579175+0000","last_undegraded":"2023-12-11T19:32:10.579175+0000","last_fullsized":"2023-12-11T19:32:10.579175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581870+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:32:10.581870+0000","last_peered":"2023-12-11T19:32:10.581870+0000","last_clean":"2023-12-11T19:32:10.581870+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:32:10.581870+0000","last_undegraded":"2023-12-11T19:32:10.581870+0000","last_fullsized":"2023-12-11T19:32:10.581870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582888+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:32:10.582888+0000","last_peered":"2023-12-11T19:32:10.582888+0000","last_clean":"2023-12-11T19:32:10.582888+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:32:10.582888+0000","last_undegraded":"2023-12-11T19:32:10.582888+0000","last_fullsized":"2023-12-11T19:32:10.582888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":150,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.567355+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:32:44.567355+0000","last_peered":"2023-12-11T19:32:44.567355+0000","last_clean":"2023-12-11T19:32:44.567355+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:32:44.567355+0000","last_undegraded":"2023-12-11T19:32:44.567355+0000","last_fullsized":"2023-12-11T19:32:44.567355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60009,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":111,"num_write":31,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584887+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:32:10.584887+0000","last_peered":"2023-12-11T19:32:10.584887+0000","last_clean":"2023-12-11T19:32:10.584887+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:32:10.584887+0000","last_undegraded":"2023-12-11T19:32:10.584887+0000","last_fullsized":"2023-12-11T19:32:10.584887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.587382+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:32:10.587382+0000","last_peered":"2023-12-11T19:32:10.587382+0000","last_clean":"2023-12-11T19:32:10.587382+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:32:10.587382+0000","last_undegraded":"2023-12-11T19:32:10.587382+0000","last_fullsized":"2023-12-11T19:32:10.587382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581911+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:32:10.581911+0000","last_peered":"2023-12-11T19:32:10.581911+0000","last_clean":"2023-12-11T19:32:10.581911+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:32:10.581911+0000","last_undegraded":"2023-12-11T19:32:10.581911+0000","last_fullsized":"2023-12-11T19:32:10.581911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581282+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:32:10.581282+0000","last_peered":"2023-12-11T19:32:10.581282+0000","last_clean":"2023-12-11T19:32:10.581282+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:32:10.581282+0000","last_undegraded":"2023-12-11T19:32:10.581282+0000","last_fullsized":"2023-12-11T19:32:10.581282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578263+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:32:10.578263+0000","last_peered":"2023-12-11T19:32:10.578263+0000","last_clean":"2023-12-11T19:32:10.578263+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:32:10.578263+0000","last_undegraded":"2023-12-11T19:32:10.578263+0000","last_fullsized":"2023-12-11T19:32:10.578263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:46.643623+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:32:46.643623+0000","last_peered":"2023-12-11T19:32:46.643623+0000","last_clean":"2023-12-11T19:32:46.643623+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:32:46.643623+0000","last_undegraded":"2023-12-11T19:32:46.643623+0000","last_fullsized":"2023-12-11T19:32:46.643623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.577975+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:32:10.577975+0000","last_peered":"2023-12-11T19:32:10.577975+0000","last_clean":"2023-12-11T19:32:10.577975+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:32:10.577975+0000","last_undegraded":"2023-12-11T19:32:10.577975+0000","last_fullsized":"2023-12-11T19:32:10.577975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583186+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:32:10.583186+0000","last_peered":"2023-12-11T19:32:10.583186+0000","last_clean":"2023-12-11T19:32:10.583186+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:32:10.583186+0000","last_undegraded":"2023-12-11T19:32:10.583186+0000","last_fullsized":"2023-12-11T19:32:10.583186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581116+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:32:10.581116+0000","last_peered":"2023-12-11T19:32:10.581116+0000","last_clean":"2023-12-11T19:32:10.581116+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:32:10.581116+0000","last_undegraded":"2023-12-11T19:32:10.581116+0000","last_fullsized":"2023-12-11T19:32:10.581116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585021+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:32:10.585021+0000","last_peered":"2023-12-11T19:32:10.585021+0000","last_clean":"2023-12-11T19:32:10.585021+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:32:10.585021+0000","last_undegraded":"2023-12-11T19:32:10.585021+0000","last_fullsized":"2023-12-11T19:32:10.585021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583296+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:32:10.583296+0000","last_peered":"2023-12-11T19:32:10.583296+0000","last_clean":"2023-12-11T19:32:10.583296+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:32:10.583296+0000","last_undegraded":"2023-12-11T19:32:10.583296+0000","last_fullsized":"2023-12-11T19:32:10.583296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583392+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:32:10.583392+0000","last_peered":"2023-12-11T19:32:10.583392+0000","last_clean":"2023-12-11T19:32:10.583392+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:32:10.583392+0000","last_undegraded":"2023-12-11T19:32:10.583392+0000","last_fullsized":"2023-12-11T19:32:10.583392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585711+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:32:10.585711+0000","last_peered":"2023-12-11T19:32:10.585711+0000","last_clean":"2023-12-11T19:32:10.585711+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:32:10.585711+0000","last_undegraded":"2023-12-11T19:32:10.585711+0000","last_fullsized":"2023-12-11T19:32:10.585711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:42.624744+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:32:42.624744+0000","last_peered":"2023-12-11T19:32:42.624744+0000","last_clean":"2023-12-11T19:32:42.624744+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:32:42.624744+0000","last_undegraded":"2023-12-11T19:32:42.624744+0000","last_fullsized":"2023-12-11T19:32:42.624744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585788+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:32:10.585788+0000","last_peered":"2023-12-11T19:32:10.585788+0000","last_clean":"2023-12-11T19:32:10.585788+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:32:10.585788+0000","last_undegraded":"2023-12-11T19:32:10.585788+0000","last_fullsized":"2023-12-11T19:32:10.585788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.594663+0000","last_change":"2023-12-11T19:32:11.594139+0000","last_active":"2023-12-11T19:32:11.594663+0000","last_peered":"2023-12-11T19:32:11.594663+0000","last_clean":"2023-12-11T19:32:11.594663+0000","last_became_active":"2023-12-11T19:32:11.593876+0000","last_became_peered":"2023-12-11T19:32:11.593876+0000","last_unstale":"2023-12-11T19:32:11.594663+0000","last_undegraded":"2023-12-11T19:32:11.594663+0000","last_fullsized":"2023-12-11T19:32:11.594663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:22:15.243111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.639020+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:32:44.639020+0000","last_peered":"2023-12-11T19:32:44.639020+0000","last_clean":"2023-12-11T19:32:44.639020+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:32:44.639020+0000","last_undegraded":"2023-12-11T19:32:44.639020+0000","last_fullsized":"2023-12-11T19:32:44.639020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585233+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:32:10.585233+0000","last_peered":"2023-12-11T19:32:10.585233+0000","last_clean":"2023-12-11T19:32:10.585233+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:32:10.585233+0000","last_undegraded":"2023-12-11T19:32:10.585233+0000","last_fullsized":"2023-12-11T19:32:10.585233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582019+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:32:10.582019+0000","last_peered":"2023-12-11T19:32:10.582019+0000","last_clean":"2023-12-11T19:32:10.582019+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:32:10.582019+0000","last_undegraded":"2023-12-11T19:32:10.582019+0000","last_fullsized":"2023-12-11T19:32:10.582019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578778+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:32:10.578778+0000","last_peered":"2023-12-11T19:32:10.578778+0000","last_clean":"2023-12-11T19:32:10.578778+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:32:10.578778+0000","last_undegraded":"2023-12-11T19:32:10.578778+0000","last_fullsized":"2023-12-11T19:32:10.578778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.590972+0000","last_change":"2023-12-11T19:32:11.590972+0000","last_active":"2023-12-11T19:32:11.590972+0000","last_peered":"2023-12-11T19:32:11.590972+0000","last_clean":"2023-12-11T19:32:11.590972+0000","last_became_active":"2023-12-11T19:32:11.590564+0000","last_became_peered":"2023-12-11T19:32:11.590564+0000","last_unstale":"2023-12-11T19:32:11.590972+0000","last_undegraded":"2023-12-11T19:32:11.590972+0000","last_fullsized":"2023-12-11T19:32:11.590972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:48:35.323725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578698+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:32:10.578698+0000","last_peered":"2023-12-11T19:32:10.578698+0000","last_clean":"2023-12-11T19:32:10.578698+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:32:10.578698+0000","last_undegraded":"2023-12-11T19:32:10.578698+0000","last_fullsized":"2023-12-11T19:32:10.578698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584892+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:32:10.584892+0000","last_peered":"2023-12-11T19:32:10.584892+0000","last_clean":"2023-12-11T19:32:10.584892+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:32:10.584892+0000","last_undegraded":"2023-12-11T19:32:10.584892+0000","last_fullsized":"2023-12-11T19:32:10.584892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581417+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:32:10.581417+0000","last_peered":"2023-12-11T19:32:10.581417+0000","last_clean":"2023-12-11T19:32:10.581417+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:32:10.581417+0000","last_undegraded":"2023-12-11T19:32:10.581417+0000","last_fullsized":"2023-12-11T19:32:10.581417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599025+0000","last_change":"2023-12-11T19:32:11.599025+0000","last_active":"2023-12-11T19:32:11.599025+0000","last_peered":"2023-12-11T19:32:11.599025+0000","last_clean":"2023-12-11T19:32:11.599025+0000","last_became_active":"2023-12-11T19:32:11.587530+0000","last_became_peered":"2023-12-11T19:32:11.587530+0000","last_unstale":"2023-12-11T19:32:11.599025+0000","last_undegraded":"2023-12-11T19:32:11.599025+0000","last_fullsized":"2023-12-11T19:32:11.599025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:08:05.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586941+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:32:10.586941+0000","last_peered":"2023-12-11T19:32:10.586941+0000","last_clean":"2023-12-11T19:32:10.586941+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:32:10.586941+0000","last_undegraded":"2023-12-11T19:32:10.586941+0000","last_fullsized":"2023-12-11T19:32:10.586941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581975+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:32:10.581975+0000","last_peered":"2023-12-11T19:32:10.581975+0000","last_clean":"2023-12-11T19:32:10.581975+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:32:10.581975+0000","last_undegraded":"2023-12-11T19:32:10.581975+0000","last_fullsized":"2023-12-11T19:32:10.581975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578880+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:32:10.578880+0000","last_peered":"2023-12-11T19:32:10.578880+0000","last_clean":"2023-12-11T19:32:10.578880+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:32:10.578880+0000","last_undegraded":"2023-12-11T19:32:10.578880+0000","last_fullsized":"2023-12-11T19:32:10.578880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581192+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:32:10.581192+0000","last_peered":"2023-12-11T19:32:10.581192+0000","last_clean":"2023-12-11T19:32:10.581192+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:32:10.581192+0000","last_undegraded":"2023-12-11T19:32:10.581192+0000","last_fullsized":"2023-12-11T19:32:10.581192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582869+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:32:10.582869+0000","last_peered":"2023-12-11T19:32:10.582869+0000","last_clean":"2023-12-11T19:32:10.582869+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:32:10.582869+0000","last_undegraded":"2023-12-11T19:32:10.582869+0000","last_fullsized":"2023-12-11T19:32:10.582869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585191+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:32:10.585191+0000","last_peered":"2023-12-11T19:32:10.585191+0000","last_clean":"2023-12-11T19:32:10.585191+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:32:10.585191+0000","last_undegraded":"2023-12-11T19:32:10.585191+0000","last_fullsized":"2023-12-11T19:32:10.585191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581598+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:32:10.581598+0000","last_peered":"2023-12-11T19:32:10.581598+0000","last_clean":"2023-12-11T19:32:10.581598+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:32:10.581598+0000","last_undegraded":"2023-12-11T19:32:10.581598+0000","last_fullsized":"2023-12-11T19:32:10.581598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586254+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:32:10.586254+0000","last_peered":"2023-12-11T19:32:10.586254+0000","last_clean":"2023-12-11T19:32:10.586254+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:32:10.586254+0000","last_undegraded":"2023-12-11T19:32:10.586254+0000","last_fullsized":"2023-12-11T19:32:10.586254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586164+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:32:10.586164+0000","last_peered":"2023-12-11T19:32:10.586164+0000","last_clean":"2023-12-11T19:32:10.586164+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:32:10.586164+0000","last_undegraded":"2023-12-11T19:32:10.586164+0000","last_fullsized":"2023-12-11T19:32:10.586164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599949+0000","last_change":"2023-12-11T19:32:11.599116+0000","last_active":"2023-12-11T19:32:11.599949+0000","last_peered":"2023-12-11T19:32:11.599949+0000","last_clean":"2023-12-11T19:32:11.599949+0000","last_became_active":"2023-12-11T19:32:11.592106+0000","last_became_peered":"2023-12-11T19:32:11.592106+0000","last_unstale":"2023-12-11T19:32:11.599949+0000","last_undegraded":"2023-12-11T19:32:11.599949+0000","last_fullsized":"2023-12-11T19:32:11.599949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:23.285224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578288+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:32:10.578288+0000","last_peered":"2023-12-11T19:32:10.578288+0000","last_clean":"2023-12-11T19:32:10.578288+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:32:10.578288+0000","last_undegraded":"2023-12-11T19:32:10.578288+0000","last_fullsized":"2023-12-11T19:32:10.578288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:44.649432+0000","last_change":"2023-12-11T19:32:11.603249+0000","last_active":"2023-12-11T19:32:44.649432+0000","last_peered":"2023-12-11T19:32:44.649432+0000","last_clean":"2023-12-11T19:32:44.649432+0000","last_became_active":"2023-12-11T19:32:11.587129+0000","last_became_peered":"2023-12-11T19:32:11.587129+0000","last_unstale":"2023-12-11T19:32:44.649432+0000","last_undegraded":"2023-12-11T19:32:44.649432+0000","last_fullsized":"2023-12-11T19:32:44.649432+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:28:44.762360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578593+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:32:10.578593+0000","last_peered":"2023-12-11T19:32:10.578593+0000","last_clean":"2023-12-11T19:32:10.578593+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:32:10.578593+0000","last_undegraded":"2023-12-11T19:32:10.578593+0000","last_fullsized":"2023-12-11T19:32:10.578593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581433+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:32:10.581433+0000","last_peered":"2023-12-11T19:32:10.581433+0000","last_clean":"2023-12-11T19:32:10.581433+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:32:10.581433+0000","last_undegraded":"2023-12-11T19:32:10.581433+0000","last_fullsized":"2023-12-11T19:32:10.581433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.596576+0000","last_change":"2023-12-11T19:32:11.596576+0000","last_active":"2023-12-11T19:32:11.596576+0000","last_peered":"2023-12-11T19:32:11.596576+0000","last_clean":"2023-12-11T19:32:11.596576+0000","last_became_active":"2023-12-11T19:32:11.592679+0000","last_became_peered":"2023-12-11T19:32:11.592679+0000","last_unstale":"2023-12-11T19:32:11.596576+0000","last_undegraded":"2023-12-11T19:32:11.596576+0000","last_fullsized":"2023-12-11T19:32:11.596576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:26:20.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581645+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:32:10.581645+0000","last_peered":"2023-12-11T19:32:10.581645+0000","last_clean":"2023-12-11T19:32:10.581645+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:32:10.581645+0000","last_undegraded":"2023-12-11T19:32:10.581645+0000","last_fullsized":"2023-12-11T19:32:10.581645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578665+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:32:10.578665+0000","last_peered":"2023-12-11T19:32:10.578665+0000","last_clean":"2023-12-11T19:32:10.578665+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:32:10.578665+0000","last_undegraded":"2023-12-11T19:32:10.578665+0000","last_fullsized":"2023-12-11T19:32:10.578665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581698+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:32:10.581698+0000","last_peered":"2023-12-11T19:32:10.581698+0000","last_clean":"2023-12-11T19:32:10.581698+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:32:10.581698+0000","last_undegraded":"2023-12-11T19:32:10.581698+0000","last_fullsized":"2023-12-11T19:32:10.581698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586080+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:32:10.586080+0000","last_peered":"2023-12-11T19:32:10.586080+0000","last_clean":"2023-12-11T19:32:10.586080+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:32:10.586080+0000","last_undegraded":"2023-12-11T19:32:10.586080+0000","last_fullsized":"2023-12-11T19:32:10.586080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586165+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:32:10.586165+0000","last_peered":"2023-12-11T19:32:10.586165+0000","last_clean":"2023-12-11T19:32:10.586165+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:32:10.586165+0000","last_undegraded":"2023-12-11T19:32:10.586165+0000","last_fullsized":"2023-12-11T19:32:10.586165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.586739+0000","last_change":"2023-12-11T19:32:11.586739+0000","last_active":"2023-12-11T19:32:11.586739+0000","last_peered":"2023-12-11T19:32:11.586739+0000","last_clean":"2023-12-11T19:32:11.586739+0000","last_became_active":"2023-12-11T19:32:11.586457+0000","last_became_peered":"2023-12-11T19:32:11.586457+0000","last_unstale":"2023-12-11T19:32:11.586739+0000","last_undegraded":"2023-12-11T19:32:11.586739+0000","last_fullsized":"2023-12-11T19:32:11.586739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:50.986234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582884+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:32:10.582884+0000","last_peered":"2023-12-11T19:32:10.582884+0000","last_clean":"2023-12-11T19:32:10.582884+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:32:10.582884+0000","last_undegraded":"2023-12-11T19:32:10.582884+0000","last_fullsized":"2023-12-11T19:32:10.582884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583047+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:32:10.583047+0000","last_peered":"2023-12-11T19:32:10.583047+0000","last_clean":"2023-12-11T19:32:10.583047+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:32:10.583047+0000","last_undegraded":"2023-12-11T19:32:10.583047+0000","last_fullsized":"2023-12-11T19:32:10.583047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586827+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:32:10.586827+0000","last_peered":"2023-12-11T19:32:10.586827+0000","last_clean":"2023-12-11T19:32:10.586827+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:32:10.586827+0000","last_undegraded":"2023-12-11T19:32:10.586827+0000","last_fullsized":"2023-12-11T19:32:10.586827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586532+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:32:10.586532+0000","last_peered":"2023-12-11T19:32:10.586532+0000","last_clean":"2023-12-11T19:32:10.586532+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:32:10.586532+0000","last_undegraded":"2023-12-11T19:32:10.586532+0000","last_fullsized":"2023-12-11T19:32:10.586532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581431+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:32:10.581431+0000","last_peered":"2023-12-11T19:32:10.581431+0000","last_clean":"2023-12-11T19:32:10.581431+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:32:10.581431+0000","last_undegraded":"2023-12-11T19:32:10.581431+0000","last_fullsized":"2023-12-11T19:32:10.581431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585160+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:32:10.585160+0000","last_peered":"2023-12-11T19:32:10.585160+0000","last_clean":"2023-12-11T19:32:10.585160+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:32:10.585160+0000","last_undegraded":"2023-12-11T19:32:10.585160+0000","last_fullsized":"2023-12-11T19:32:10.585160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582513+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:32:10.582513+0000","last_peered":"2023-12-11T19:32:10.582513+0000","last_clean":"2023-12-11T19:32:10.582513+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:32:10.582513+0000","last_undegraded":"2023-12-11T19:32:10.582513+0000","last_fullsized":"2023-12-11T19:32:10.582513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581557+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:32:10.581557+0000","last_peered":"2023-12-11T19:32:10.581557+0000","last_clean":"2023-12-11T19:32:10.581557+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:32:10.581557+0000","last_undegraded":"2023-12-11T19:32:10.581557+0000","last_fullsized":"2023-12-11T19:32:10.581557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582438+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:32:10.582438+0000","last_peered":"2023-12-11T19:32:10.582438+0000","last_clean":"2023-12-11T19:32:10.582438+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:32:10.582438+0000","last_undegraded":"2023-12-11T19:32:10.582438+0000","last_fullsized":"2023-12-11T19:32:10.582438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67209,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":493,"num_read_kb":325,"num_write":230,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2181,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":201627,"data_compressed":5496,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":459769,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.83199999999999996}]},{"osd":2,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.79400000000000004}]},{"osd":5,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.91000000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.0980000000000001}]},{"osd":7,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.92400000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":9,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.0860000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.1819999999999999}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1111202,"data_compressed":43838,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.61599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.503},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91200000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54300000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82499999999999996,"5min":1.04,"15min":1.04},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.442,"15min":0.442},"max":{"1min":0.94699999999999995,"5min":1.032,"15min":1.032},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.496,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.61299999999999999}]},{"osd":4,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41399999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.69999999999999996,"15min":0.71299999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92300000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98499999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.80300000000000005,"15min":0.81299999999999994},"min":{"1min":0.54900000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.085,"15min":1.085},"last":0.98399999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.85299999999999998,"15min":0.83899999999999997},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.83399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0089999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":0.90500000000000003},"min":{"1min":0.66500000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0640000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.92200000000000004,"15min":0.93799999999999994},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0609999999999999,"5min":1.643,"15min":1.643},"last":0.90800000000000003}]},{"osd":9,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90400000000000003,"15min":0.90800000000000003},"min":{"1min":0.57899999999999996,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.034,"5min":1.321,"15min":1.321},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0309999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.747}]},{"osd":11,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0009999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":459822,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.49399999999999999}]},{"osd":1,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.83899999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0609999999999999,"5min":1.349,"15min":1.349},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.44},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74299999999999999,"15min":0.753},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90300000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.62}]},{"osd":4,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.83499999999999996,"15min":0.84899999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.90800000000000003,"15min":0.92500000000000004},"min":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":1.278,"15min":1.278},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.90100000000000002,"15min":0.91500000000000004},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.115,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0429999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90500000000000003,"15min":0.93200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.389,"15min":1.389},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.96799999999999997,"15min":0.98399999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0960000000000001,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.871},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.97299999999999998,"15min":0.98599999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.2250000000000001,"5min":1.274,"15min":1.274},"last":0.84399999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.0009999999999999,"15min":1.018},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1850000000000001,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.0089999999999999,"15min":1.0249999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.097,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.91800000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.98599999999999999,"15min":1.0069999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0549999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.98599999999999999,"15min":0.998},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.169,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":460077,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.751},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":1.097,"15min":1.097},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.55700000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1479999999999999,"5min":1.351,"15min":1.351},"last":0.996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.81200000000000006,"15min":0.80900000000000005},"min":{"1min":0.61399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.0740000000000001,"5min":2.21,"15min":2.21},"last":0.80000000000000004}]},{"osd":2,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.622},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.292,"15min":1.292},"last":0.77000000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.2350000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.58199999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1140000000000001,"5min":2.173,"15min":2.173},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.083,"5min":1.177,"15min":1.177},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.84899999999999998,"15min":0.82099999999999995},"min":{"1min":0.74199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2869999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1459999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.84899999999999998}]},{"osd":7,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.71699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.304,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.74099999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.95199999999999996}]},{"osd":9,"last update":"Mon Dec 11 19:32:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.92100000000000004,"15min":0.91000000000000003},"min":{"1min":0.71599999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.4470000000000001,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.751,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.083}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.79200000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.405,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.056,"5min":0.98299999999999998,"15min":0.97099999999999997},"min":{"1min":0.77600000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.343,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.1180000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1052657,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.88500000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80600000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.94999999999999996}]},{"osd":3,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.78300000000000003,"15min":0.81499999999999995},"min":{"1min":0.51000000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79700000000000004,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.72299999999999998,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.92600000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.65100000000000002}]},{"osd":4,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75700000000000001,"15min":0.78400000000000003},"min":{"1min":0.51200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94699999999999995,"5min":1.238,"15min":1.238},"last":0.61199999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98599999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85699999999999998,"15min":0.86299999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1000000000000001,"5min":1.28,"15min":1.28},"last":0.87}]},{"osd":6,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0800000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77700000000000002,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.052,"5min":1.359,"15min":1.359},"last":0.63600000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97999999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.85999999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.248,"15min":1.248},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.89500000000000002,"15min":0.91500000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.95799999999999996,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0920000000000001,"5min":1.25,"15min":1.25},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":459422,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.315,"5min":0.31,"15min":0.31},"max":{"1min":0.68400000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.83499999999999996}]},{"osd":2,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92500000000000004,"5min":1.284,"15min":1.284},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95599999999999996,"5min":1.228,"15min":1.228},"last":0.66100000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.016,"15min":1.016},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.119,"5min":1.157,"15min":1.157},"last":0.91600000000000004}]},{"osd":5,"last update":"Mon Dec 11 19:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82399999999999995,"15min":0.81200000000000006},"min":{"1min":0.65200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.88600000000000001,"15min":0.92200000000000004},"min":{"1min":0.65400000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0349999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Dec 11 19:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.81799999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.089,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.879,"15min":0.86799999999999999},"min":{"1min":0.629,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.181,"5min":1.234,"15min":1.234},"last":0.95699999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.879},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.80100000000000005}]},{"osd":9,"last update":"Mon Dec 11 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.92200000000000004},"min":{"1min":0.67100000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.86699999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1859999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.85499999999999998}]},{"osd":10,"last update":"Mon Dec 11 19:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.84899999999999998,"15min":0.83299999999999996},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.012,"5min":0.98099999999999998,"15min":0.97599999999999998},"min":{"1min":0.84899999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.98399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521235,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.55700000000000005},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69099999999999995,"15min":0.58199999999999996},"min":{"1min":0.58099999999999996,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":1.05,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.70599999999999996},"min":{"1min":0.47899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.081,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.65600000000000003},"min":{"1min":0.50900000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1779999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.81899999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.50900000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.53000000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91100000000000003,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.629}]},{"osd":4,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.437},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.60399999999999998,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Dec 11 19:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.70799999999999996,"15min":0.66800000000000004},"min":{"1min":0.56100000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.135,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.77000000000000002,"15min":0.86699999999999999},"min":{"1min":0.54700000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92800000000000005,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Dec 11 19:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.63,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.05,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Dec 11 19:32:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.69699999999999995,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64600000000000002,"15min":0.61599999999999999},"min":{"1min":0.57199999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.501}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2110000000000001,"5min":1.52,"15min":1.52},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.091,"5min":0.88500000000000001,"15min":0.83799999999999997},"min":{"1min":0.67100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.95999999999999996}]},{"osd":9,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.88400000000000001,"15min":0.86399999999999999},"min":{"1min":0.79000000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.89000000000000001,"15min":0.875},"min":{"1min":0.752,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.84499999999999997}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.83599999999999997,"15min":0.82799999999999996},"min":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0840000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93300000000000005},"min":{"1min":0.76400000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.196,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":460755,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82799999999999996,"15min":0.67800000000000005},"min":{"1min":0.52300000000000002,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":1.1890000000000001,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":1.0449999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71499999999999997,"15min":0.63500000000000001},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1120000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.439,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94299999999999995,"5min":1.403,"15min":1.403},"last":0.67600000000000005}]},{"osd":3,"last update":"Mon Dec 11 19:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.46100000000000002},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77200000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":1.004,"15min":1.004},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0389999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.48999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Dec 11 19:32:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89800000000000002,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.95099999999999996,"5min":1.819,"15min":1.819},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Dec 11 19:31:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.80300000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.90400000000000003,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1579999999999999,"5min":1.25,"15min":1.25},"last":0.92800000000000005}]},{"osd":9,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.98499999999999999,"15min":1.014},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3049999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.93100000000000005,"15min":0.96299999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96899999999999997,"5min":1.202,"15min":1.202},"last":0.88500000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.86499999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":518766,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.502},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.44400000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Dec 11 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.63700000000000001},"min":{"1min":0.40300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95199999999999996,"5min":1.115,"15min":1.115},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.69099999999999995,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.98999999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82799999999999996}]},{"osd":3,"last update":"Mon Dec 11 19:32:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.67700000000000005,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.0249999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Dec 11 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.58999999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.86399999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.59799999999999998,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93400000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.39300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.627}]},{"osd":6,"last update":"Mon Dec 11 19:31:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.58499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.747,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.90100000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.84099999999999997,"15min":0.82699999999999996},"min":{"1min":0.64500000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.877,"15min":0.86499999999999999},"min":{"1min":0.70599999999999996,"5min":0.628,"15min":0.628},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.91900000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.90400000000000003,"15min":0.89200000000000002},"min":{"1min":0.73399999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.996,"5min":0.90000000000000002,"15min":0.88400000000000001},"min":{"1min":0.78400000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0129999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":458799,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.40500000000000003},"min":{"1min":0.38900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.501,"15min":0.435},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65800000000000003,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.68799999999999994,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.63800000000000001},"min":{"1min":0.57099999999999995,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.64300000000000002},"min":{"1min":0.503,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Dec 11 19:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.054,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.75600000000000001,"15min":0.69999999999999996},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.105,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.746}]},{"osd":5,"last update":"Mon Dec 11 19:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.53100000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.76100000000000001,"15min":0.73899999999999999},"min":{"1min":0.55200000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72799999999999998,"15min":0.71299999999999997},"min":{"1min":0.48699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:32:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.82599999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86099999999999999,"15min":0.83799999999999997},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.621},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.82899999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.96099999999999997,"15min":0.97099999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0089999999999999,"15min":1.02},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.347,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.627}]},{"osd":10,"last update":"Mon Dec 11 19:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.98699999999999999,"15min":0.98399999999999999},"min":{"1min":0.79600000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.056,"15min":1.0669999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":460677,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.47199999999999998,"15min":0.46400000000000002},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.85199999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.48299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.879}]},{"osd":3,"last update":"Mon Dec 11 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67900000000000005,"15min":0.65200000000000002},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94999999999999996,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64700000000000002,"15min":0.63300000000000001},"min":{"1min":0.45600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68500000000000005}]},{"osd":5,"last update":"Mon Dec 11 19:32:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73699999999999999,"15min":0.69699999999999995},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.361,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.72699999999999998,"15min":0.71699999999999997},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.748}]},{"osd":6,"last update":"Mon Dec 11 19:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72999999999999998,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.395,"5min":1.429,"15min":1.429},"last":0.91600000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:32:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72299999999999998,"15min":0.70699999999999996},"min":{"1min":0.439,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.70899999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.67600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.94799999999999995}]},{"osd":9,"last update":"Mon Dec 11 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88700000000000001,"15min":0.88300000000000001},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4399999999999999,"5min":1.494,"15min":1.494},"last":1.002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.81699999999999995}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90200000000000002,"15min":0.89900000000000002},"min":{"1min":0.68799999999999994,"5min":0.628,"15min":0.628},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.95599999999999996,"15min":0.95899999999999996},"min":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5029999999999999,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.075}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]}]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":1052277,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.629,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.63100000000000001,"15min":0.65900000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90000000000000002,"5min":1.177,"15min":1.177},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67700000000000005,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0409999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85899999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.502,"15min":0.46600000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86899999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.754},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72599999999999998,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.78200000000000003}]},{"osd":4,"last update":"Mon Dec 11 19:32:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.51400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.141,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71499999999999997,"15min":0.74099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76300000000000001,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.746},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.248,"5min":1.597,"15min":1.597},"last":0.83899999999999997}]},{"osd":7,"last update":"Mon Dec 11 19:32:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76400000000000001,"15min":0.745},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1659999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.999},{"interface":"front","average":{"1min":0.879,"5min":0.81699999999999995,"15min":0.80100000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.02}]},{"osd":8,"last update":"Mon Dec 11 19:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.214,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.0740000000000001}]},{"osd":9,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.61499999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1819999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.88500000000000001,"15min":0.88900000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1759999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.05}]},{"osd":10,"last update":"Mon Dec 11 19:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90200000000000002,"15min":0.90500000000000003},"min":{"1min":0.66900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.256,"5min":1.548,"15min":1.548},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92900000000000005,"15min":0.92600000000000005},"min":{"1min":0.63700000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.1220000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62931,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60189,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62134,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:32:50.411 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-11T19:32:50.411 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-11T19:32:50.411 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-11T19:32:50.411 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph health --format=json 2023-12-11T19:32:51.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:50 smithi167 ceph-mon[102750]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:51.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:50 smithi066 ceph-mon[97920]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:51.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:50 smithi070 ceph-mon[104044]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:32:51.210 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:32:51.802 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:51 smithi066 ceph-mon[97920]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:32:52.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:51 smithi167 ceph-mon[102750]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:32:52.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:51 smithi070 ceph-mon[104044]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:32:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.733+0000 7f61fc798700 1 -- 172.21.15.66:0/4222473724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 msgr2=0x7f61f4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.733+0000 7f61fc798700 1 --2- 172.21.15.66:0/4222473724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f4069350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009990 tx=0x7f61f0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 -- 172.21.15.66:0/4222473724 shutdown_connections 2023-12-11T19:32:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 --2- 172.21.15.66:0/4222473724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61f410e0e0 0x7f61f41104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 --2- 172.21.15.66:0/4222473724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f4069350 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 --2- 172.21.15.66:0/4222473724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4069890 0x7f61f410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 -- 172.21.15.66:0/4222473724 >> 172.21.15.66:0/4222473724 conn(0x7f61f4076570 msgr2=0x7f61f4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 -- 172.21.15.66:0/4222473724 shutdown_connections 2023-12-11T19:32:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.734+0000 7f61fc798700 1 -- 172.21.15.66:0/4222473724 wait complete. 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 Processor -- start 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 -- start start 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4069890 0x7f61f419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61f410e0e0 0x7f61f41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419a030 con 0x7f61f4068f70 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419a1a0 con 0x7f61f4069890 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.735+0000 7f61fc798700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419a360 con 0x7f61f410e0e0 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49242/0 (socket says 172.21.15.66:49242) 2023-12-11T19:32:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 -- 172.21.15.66:0/845774736 learned_addr learned my addr 172.21.15.66:0/845774736 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61f9d33700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4069890 0x7f61f419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fad35700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61f410e0e0 0x7f61f41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 -- 172.21.15.66:0/845774736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61f410e0e0 msgr2=0x7f61f41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61f410e0e0 0x7f61f41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 -- 172.21.15.66:0/845774736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4069890 msgr2=0x7f61f419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4069890 0x7f61f419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 -- 172.21.15.66:0/845774736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00095f0 con 0x7f61f4068f70 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.736+0000 7f61fa534700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f419f890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61f00098e0 tx=0x7f61f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.737+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f001d070 con 0x7f61f4068f70 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.737+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f000fb00 con 0x7f61f4068f70 2023-12-11T19:32:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.737+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0022aa0 con 0x7f61f4068f70 2023-12-11T19:32:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.737+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f419a5e0 con 0x7f61f4068f70 2023-12-11T19:32:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.737+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f41a6880 con 0x7f61f4068f70 2023-12-11T19:32:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.738+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f4109300 con 0x7f61f4068f70 2023-12-11T19:32:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.740+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f61f000fc70 con 0x7f61f4068f70 2023-12-11T19:32:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.741+0000 7f61eb7fe700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61e00a1590 0x7f61e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.741+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f61f00ce340 con 0x7f61f4068f70 2023-12-11T19:32:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.742+0000 7f61f9d33700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61e00a1590 0x7f61e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.742+0000 7f61f9d33700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61e00a1590 0x7f61e00a3a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f61f419ab90 tx=0x7f61e40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:52.746+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f61f0097890 con 0x7f61f4068f70 2023-12-11T19:32:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.002+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f61f419ae90 con 0x7f61f4068f70 2023-12-11T19:32:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.002+0000 7f61eb7fe700 1 -- 172.21.15.66:0/845774736 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f61f009aeb0 con 0x7f61f4068f70 2023-12-11T19:32:53.004 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:53.004 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.004+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61e00a1590 msgr2=0x7f61e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.004+0000 7f61fc798700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61e00a1590 0x7f61e00a3a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f61f419ab90 tx=0x7f61e40073d0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 msgr2=0x7f61f419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f419f890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f61f00098e0 tx=0x7f61f0005740 comp rx=0 tx=0).stop 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 shutdown_connections 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61e00a1590 0x7f61e00a3a50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61f410e0e0 0x7f61f41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61f4068f70 0x7f61f419f890 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 --2- 172.21.15.66:0/845774736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4069890 0x7f61f419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 >> 172.21.15.66:0/845774736 conn(0x7f61f4076570 msgr2=0x7f61f410f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:53.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 shutdown_connections 2023-12-11T19:32:53.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.005+0000 7f61fc798700 1 -- 172.21.15.66:0/845774736 wait complete. 2023-12-11T19:32:53.510 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-11T19:32:53.511 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 -- 172.21.15.66:0/722095529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 msgr2=0x7f89181098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 --2- 172.21.15.66:0/722095529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f89181098e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f891000b0d0 tx=0x7f891000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 -- 172.21.15.66:0/722095529 shutdown_connections 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 --2- 172.21.15.66:0/722095529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f89181098e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 --2- 172.21.15.66:0/722095529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89181014a0 0x7f89181018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 --2- 172.21.15.66:0/722095529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8918102690 0x7f8918102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 -- 172.21.15.66:0/722095529 >> 172.21.15.66:0/722095529 conn(0x7f89180fca10 msgr2=0x7f89180fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 -- 172.21.15.66:0/722095529 shutdown_connections 2023-12-11T19:32:53.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.722+0000 7f891ffff700 1 -- 172.21.15.66:0/722095529 wait complete. 2023-12-11T19:32:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.723+0000 7f891ffff700 1 Processor -- start 2023-12-11T19:32:53.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 -- start start 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89181014a0 0x7f8918197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8918102690 0x7f8918197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f891819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891819ef80 con 0x7f8918103030 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891819f0c0 con 0x7f8918102690 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891819f250 con 0x7f89181014a0 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f891819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f891819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49260/0 (socket says 172.21.15.66:49260) 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891e59c700 1 -- 172.21.15.66:0/718177200 learned_addr learned my addr 172.21.15.66:0/718177200 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891dd9b700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89181014a0 0x7f8918197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.724+0000 7f891d59a700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8918102690 0x7f8918197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891e59c700 1 -- 172.21.15.66:0/718177200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89181014a0 msgr2=0x7f8918197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891e59c700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89181014a0 0x7f8918197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891e59c700 1 -- 172.21.15.66:0/718177200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8918102690 msgr2=0x7f8918197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891e59c700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8918102690 0x7f8918197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891e59c700 1 -- 172.21.15.66:0/718177200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910009d00 con 0x7f8918103030 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891e59c700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f891819e9b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8910007b60 tx=0x7f8910009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000e040 con 0x7f8918103030 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8910012d90 con 0x7f8918103030 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891819f4d0 con 0x7f8918103030 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001de80 con 0x7f8918103030 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.725+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891819f9a0 con 0x7f8918103030 2023-12-11T19:32:53.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.726+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8918108920 con 0x7f8918103030 2023-12-11T19:32:53.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.728+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f8910012900 con 0x7f8918103030 2023-12-11T19:32:53.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.731+0000 7f890effd700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f89040a1590 0x7f89040a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:53.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.731+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f89100cec60 con 0x7f8918103030 2023-12-11T19:32:53.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.731+0000 7f891dd9b700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f89040a1590 0x7f89040a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:53.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.731+0000 7f891dd9b700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f89040a1590 0x7f89040a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f89181024f0 tx=0x7f891400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:53.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.735+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f89100981b0 con 0x7f8918103030 2023-12-11T19:32:53.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.953+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f891810cbd0 con 0x7f8918103030 2023-12-11T19:32:53.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.955+0000 7f890effd700 1 -- 172.21.15.66:0/718177200 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f891009b7d0 con 0x7f8918103030 2023-12-11T19:32:53.956 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:53.957 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.957+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f89040a1590 msgr2=0x7f89040a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.957+0000 7f891ffff700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f89040a1590 0x7f89040a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f89181024f0 tx=0x7f891400b410 comp rx=0 tx=0).stop 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.957+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 msgr2=0x7f891819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.957+0000 7f891ffff700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f891819e9b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8910007b60 tx=0x7f8910009440 comp rx=0 tx=0).stop 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 shutdown_connections 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f89040a1590 0x7f89040a3a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8918103030 0x7f891819e9b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89181014a0 0x7f8918197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 --2- 172.21.15.66:0/718177200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8918102690 0x7f8918197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 >> 172.21.15.66:0/718177200 conn(0x7f89180fca10 msgr2=0x7f8918105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 shutdown_connections 2023-12-11T19:32:53.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:53.958+0000 7f891ffff700 1 -- 172.21.15.66:0/718177200 wait complete. 2023-12-11T19:32:53.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:53 smithi066 ceph-mon[97920]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:53.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:53 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/845774736' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-11T19:32:53.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:32:53.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:32:54.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:53 smithi167 ceph-mon[102750]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:54.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:53 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/845774736' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-11T19:32:54.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:53 smithi070 ceph-mon[104044]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:54.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:53 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/845774736' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-11T19:32:54.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 -- 172.21.15.66:0/2341246416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 msgr2=0x7fbda4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/2341246416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4102bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd940099c0 tx=0x7fbd94009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 -- 172.21.15.66:0/2341246416 shutdown_connections 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/2341246416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda4103130 0x7fbda41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/2341246416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda41015a0 0x7fbda41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/2341246416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4102bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 -- 172.21.15.66:0/2341246416 >> 172.21.15.66:0/2341246416 conn(0x7fbda40fcb10 msgr2=0x7fbda40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 -- 172.21.15.66:0/2341246416 shutdown_connections 2023-12-11T19:32:54.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.189+0000 7fbdab3ff700 1 -- 172.21.15.66:0/2341246416 wait complete. 2023-12-11T19:32:54.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.190+0000 7fbdab3ff700 1 Processor -- start 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.190+0000 7fbdab3ff700 1 -- start start 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.190+0000 7fbdab3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda41015a0 0x7fbda41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbdab3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbdab3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda4103130 0x7fbda41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbdab3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41a11b0 con 0x7fbda4102790 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbdab3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41a12f0 con 0x7fbda4103130 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbdab3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41a1480 con 0x7fbda41015a0 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49282/0 (socket says 172.21.15.66:49282) 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 -- 172.21.15.66:0/928535281 learned_addr learned my addr 172.21.15.66:0/928535281 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda919b700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda41015a0 0x7fbda41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda999c700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda4103130 0x7fbda41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 -- 172.21.15.66:0/928535281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda41015a0 msgr2=0x7fbda41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda41015a0 0x7fbda41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 -- 172.21.15.66:0/928535281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda4103130 msgr2=0x7fbda41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda4103130 0x7fbda41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.191+0000 7fbda899a700 1 -- 172.21.15.66:0/928535281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940095f0 con 0x7fbda4102790 2023-12-11T19:32:54.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.192+0000 7fbda899a700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4199c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009990 tx=0x7fbd9400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:54.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.192+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9401d070 con 0x7fbda4102790 2023-12-11T19:32:54.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.192+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd9400bd10 con 0x7fbda4102790 2023-12-11T19:32:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.192+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda41a1700 con 0x7fbda4102790 2023-12-11T19:32:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.192+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94017650 con 0x7fbda4102790 2023-12-11T19:32:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.192+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda41a1b80 con 0x7fbda4102790 2023-12-11T19:32:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.193+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda4068310 con 0x7fbda4102790 2023-12-11T19:32:54.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.195+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbd94022470 con 0x7fbda4102790 2023-12-11T19:32:54.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.196+0000 7fbd9a7fc700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd900a1590 0x7fbd900a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.196+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd940ce0f0 con 0x7fbda4102790 2023-12-11T19:32:54.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.197+0000 7fbda919b700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd900a1590 0x7fbd900a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.197+0000 7fbda919b700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd900a1590 0x7fbd900a3a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbda41025f0 tx=0x7fbda0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:54.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.200+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbd94097640 con 0x7fbda4102790 2023-12-11T19:32:54.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.414+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbda41a1e40 con 0x7fbda4102790 2023-12-11T19:32:54.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.414+0000 7fbd9a7fc700 1 -- 172.21.15.66:0/928535281 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd9409ac60 con 0x7fbda4102790 2023-12-11T19:32:54.416 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:32:54.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd900a1590 msgr2=0x7fbd900a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd900a1590 0x7fbd900a3a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbda41025f0 tx=0x7fbda0009260 comp rx=0 tx=0).stop 2023-12-11T19:32:54.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 msgr2=0x7fbda4199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4199c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009990 tx=0x7fbd9400fab0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 shutdown_connections 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd900a1590 0x7fbd900a3a50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda4103130 0x7fbda41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda41015a0 0x7fbda41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 --2- 172.21.15.66:0/928535281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda4102790 0x7fbda4199c10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 >> 172.21.15.66:0/928535281 conn(0x7fbda40fcb10 msgr2=0x7fbda4073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 shutdown_connections 2023-12-11T19:32:54.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.417+0000 7fbdab3ff700 1 -- 172.21.15.66:0/928535281 wait complete. 2023-12-11T19:32:54.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:32:54.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.647+0000 7fab789ff700 1 -- 172.21.15.66:0/4204960055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab741072c0 msgr2=0x7fab741096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.647+0000 7fab789ff700 1 --2- 172.21.15.66:0/4204960055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab741072c0 0x7fab741096d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b0a0 tx=0x7fab6c00b3b0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.647+0000 7fab789ff700 1 -- 172.21.15.66:0/4204960055 shutdown_connections 2023-12-11T19:32:54.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.648+0000 7fab789ff700 1 --2- 172.21.15.66:0/4204960055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74102060 0x7fab74104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.648+0000 7fab789ff700 1 --2- 172.21.15.66:0/4204960055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab741072c0 0x7fab741096d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.648+0000 7fab789ff700 1 --2- 172.21.15.66:0/4204960055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74104990 0x7fab74106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.648+0000 7fab789ff700 1 -- 172.21.15.66:0/4204960055 >> 172.21.15.66:0/4204960055 conn(0x7fab740fb9f0 msgr2=0x7fab740fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:54.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.648+0000 7fab789ff700 1 -- 172.21.15.66:0/4204960055 shutdown_connections 2023-12-11T19:32:54.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.648+0000 7fab789ff700 1 -- 172.21.15.66:0/4204960055 wait complete. 2023-12-11T19:32:54.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 Processor -- start 2023-12-11T19:32:54.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 -- start start 2023-12-11T19:32:54.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 0x7fab74197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74104990 0x7fab74197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab741072c0 0x7fab7419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7419f020 con 0x7fab74102060 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7419f160 con 0x7fab74104990 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab789ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7419f2f0 con 0x7fab741072c0 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab7259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 0x7fab74197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab7259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 0x7fab74197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49294/0 (socket says 172.21.15.66:49294) 2023-12-11T19:32:54.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.649+0000 7fab7259c700 1 -- 172.21.15.66:0/2379961165 learned_addr learned my addr 172.21.15.66:0/2379961165 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab72d9d700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab741072c0 0x7fab7419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab71d9b700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74104990 0x7fab74197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab7259c700 1 -- 172.21.15.66:0/2379961165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab741072c0 msgr2=0x7fab7419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab7259c700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab741072c0 0x7fab7419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab7259c700 1 -- 172.21.15.66:0/2379961165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74104990 msgr2=0x7fab74197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab7259c700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74104990 0x7fab74197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab7259c700 1 -- 172.21.15.66:0/2379961165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c009d00 con 0x7fab74102060 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab7259c700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 0x7fab74197590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab6400e9a0 tx=0x7fab6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:54.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64009790 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab6400cbd0 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7419f5d0 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.650+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64018680 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.651+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7419fb30 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.651+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab740fd570 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.653+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fab6400cd40 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.655+0000 7fab637fe700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fab5c0a1590 0x7fab5c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.655+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fab64014070 con 0x7fab74102060 2023-12-11T19:32:54.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.655+0000 7fab71d9b700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fab5c0a1590 0x7fab5c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.656+0000 7fab71d9b700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fab5c0a1590 0x7fab5c0a3a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab68005fd0 tx=0x7fab68005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.658+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fab640522f0 con 0x7fab74102060 2023-12-11T19:32:54.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.875+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab7410cf40 con 0x7fab74102060 2023-12-11T19:32:54.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.875+0000 7fab637fe700 1 -- 172.21.15.66:0/2379961165 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab640965a0 con 0x7fab74102060 2023-12-11T19:32:54.877 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fab5c0a1590 msgr2=0x7fab5c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fab5c0a1590 0x7fab5c0a3a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab68005fd0 tx=0x7fab68005f60 comp rx=0 tx=0).stop 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 msgr2=0x7fab74197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 0x7fab74197590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab6400e9a0 tx=0x7fab6400ecb0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 shutdown_connections 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fab5c0a1590 0x7fab5c0a3a50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab741072c0 0x7fab7419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab74102060 0x7fab74197590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 --2- 172.21.15.66:0/2379961165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74104990 0x7fab74197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 >> 172.21.15.66:0/2379961165 conn(0x7fab740fb9f0 msgr2=0x7fab74102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 shutdown_connections 2023-12-11T19:32:54.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:54.878+0000 7fab789ff700 1 -- 172.21.15.66:0/2379961165 wait complete. 2023-12-11T19:32:54.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:32:54.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:54 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/718177200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:54.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:54 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/928535281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:32:55.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:54 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/718177200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:55.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:54 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/928535281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:32:55.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:54 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/718177200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:55.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:54 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/928535281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:32:55.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 -- 172.21.15.66:0/2317317718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 msgr2=0x7f18e4106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/2317317718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4106d20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009990 tx=0x7f18d8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 -- 172.21.15.66:0/2317317718 shutdown_connections 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/2317317718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4107260 0x7f18e41096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/2317317718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4102000 0x7f18e41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/2317317718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4106d20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 -- 172.21.15.66:0/2317317718 >> 172.21.15.66:0/2317317718 conn(0x7f18e40fb9f0 msgr2=0x7f18e40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 -- 172.21.15.66:0/2317317718 shutdown_connections 2023-12-11T19:32:55.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.109+0000 7f18ea1ff700 1 -- 172.21.15.66:0/2317317718 wait complete. 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.110+0000 7f18ea1ff700 1 Processor -- start 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.110+0000 7f18ea1ff700 1 -- start start 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18ea1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4102000 0x7f18e4197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18ea1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18ea1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4107260 0x7f18e419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18ea1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e419ef80 con 0x7f18e4104930 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18ea1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e419f0c0 con 0x7f18e4102000 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18ea1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e419f250 con 0x7f18e4107260 2023-12-11T19:32:55.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49320/0 (socket says 172.21.15.66:49320) 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 -- 172.21.15.66:0/3015104019 learned_addr learned my addr 172.21.15.66:0/3015104019 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e37fe700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4102000 0x7f18e4197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e3fff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4107260 0x7f18e419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 -- 172.21.15.66:0/3015104019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4107260 msgr2=0x7f18e419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4107260 0x7f18e419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 -- 172.21.15.66:0/3015104019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4102000 msgr2=0x7f18e4197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4102000 0x7f18e4197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e2ffd700 1 -- 172.21.15.66:0/3015104019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80095f0 con 0x7f18e4104930 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e3fff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4107260 0x7f18e419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.111+0000 7f18e37fe700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4102000 0x7f18e4197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.112+0000 7f18e2ffd700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4197990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005740 tx=0x7f18d8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.112+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d801d070 con 0x7f18e4104930 2023-12-11T19:32:55.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.112+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e419f4d0 con 0x7f18e4104930 2023-12-11T19:32:55.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.112+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18d800beb0 con 0x7f18e4104930 2023-12-11T19:32:55.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.112+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8021410 con 0x7f18e4104930 2023-12-11T19:32:55.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.112+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e419f900 con 0x7f18e4104930 2023-12-11T19:32:55.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.113+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e40fd570 con 0x7f18e4104930 2023-12-11T19:32:55.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.115+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f18d800ba20 con 0x7f18e4104930 2023-12-11T19:32:55.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.116+0000 7f18e0ff9700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18cc0a1510 0x7f18cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.116+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f18d80ce340 con 0x7f18e4104930 2023-12-11T19:32:55.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.117+0000 7f18e37fe700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18cc0a1510 0x7f18cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.117+0000 7f18e37fe700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18cc0a1510 0x7f18cc0a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005fd0 tx=0x7f18d4005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:55.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.120+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f18d8097890 con 0x7f18e4104930 2023-12-11T19:32:55.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.336+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18e410cdf0 con 0x7f18e4104930 2023-12-11T19:32:55.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.337+0000 7f18e0ff9700 1 -- 172.21.15.66:0/3015104019 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18d809aeb0 con 0x7f18e4104930 2023-12-11T19:32:55.338 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18cc0a1510 msgr2=0x7f18cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18cc0a1510 0x7f18cc0a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005fd0 tx=0x7f18d4005f60 comp rx=0 tx=0).stop 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 msgr2=0x7f18e4197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4197990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005740 tx=0x7f18d8005e80 comp rx=0 tx=0).stop 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 shutdown_connections 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18cc0a1510 0x7f18cc0a39d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e4102000 0x7f18e4197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18e4107260 0x7f18e419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 --2- 172.21.15.66:0/3015104019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e4104930 0x7f18e4197990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.339+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 >> 172.21.15.66:0/3015104019 conn(0x7f18e40fb9f0 msgr2=0x7f18e4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.340+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 shutdown_connections 2023-12-11T19:32:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.340+0000 7f18ea1ff700 1 -- 172.21.15.66:0/3015104019 wait complete. 2023-12-11T19:32:55.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 -- 172.21.15.66:0/3140050433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 msgr2=0x7f24c8102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 --2- 172.21.15.66:0/3140050433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c8102ac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f24b00099c0 tx=0x7f24b0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 -- 172.21.15.66:0/3140050433 shutdown_connections 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 --2- 172.21.15.66:0/3140050433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24c8101470 0x7f24c8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 --2- 172.21.15.66:0/3140050433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8103000 0x7f24c81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 --2- 172.21.15.66:0/3140050433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c8102ac0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 -- 172.21.15.66:0/3140050433 >> 172.21.15.66:0/3140050433 conn(0x7f24c80fc9e0 msgr2=0x7f24c80fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 -- 172.21.15.66:0/3140050433 shutdown_connections 2023-12-11T19:32:55.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.569+0000 7f24c759e700 1 -- 172.21.15.66:0/3140050433 wait complete. 2023-12-11T19:32:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.570+0000 7f24c759e700 1 Processor -- start 2023-12-11T19:32:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.570+0000 7f24c759e700 1 -- start start 2023-12-11T19:32:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24c8101470 0x7f24c819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c759e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8103000 0x7f24c81a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c759e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c81a5460 con 0x7f24c8102660 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c81a55a0 con 0x7f24c8101470 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c81a5730 con 0x7f24c8103000 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49348/0 (socket says 172.21.15.66:49348) 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 -- 172.21.15.66:0/3854572312 learned_addr learned my addr 172.21.15.66:0/3854572312 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c659c700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24c8101470 0x7f24c819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24c6d9d700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8103000 0x7f24c81a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 -- 172.21.15.66:0/3854572312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8103000 msgr2=0x7f24c81a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8103000 0x7f24c81a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 -- 172.21.15.66:0/3854572312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24c8101470 msgr2=0x7f24c819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24c8101470 0x7f24c819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.571+0000 7f24bffff700 1 -- 172.21.15.66:0/3854572312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00095f0 con 0x7f24c8102660 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.572+0000 7f24bffff700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c819de70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009990 tx=0x7f24b000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.572+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b001d070 con 0x7f24c8102660 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.572+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24b000bd10 con 0x7f24c8102660 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.572+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c81a59b0 con 0x7f24c8102660 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.572+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0017650 con 0x7f24c8102660 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.572+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c81a5e30 con 0x7f24c8102660 2023-12-11T19:32:55.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.573+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c80681c0 con 0x7f24c8102660 2023-12-11T19:32:55.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.575+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f24b0022470 con 0x7f24c8102660 2023-12-11T19:32:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.576+0000 7f24bf7fe700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24b40a1590 0x7f24b40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.576+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24b00ce200 con 0x7f24c8102660 2023-12-11T19:32:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.577+0000 7f24c659c700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24b40a1590 0x7f24b40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:55.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.577+0000 7f24c659c700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24b40a1590 0x7f24b40a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24c81024c0 tx=0x7f24b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:55.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.580+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f24b0097750 con 0x7f24c8102660 2023-12-11T19:32:55.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.848+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24c810cb90 con 0x7f24c8102660 2023-12-11T19:32:55.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.849+0000 7f24bf7fe700 1 -- 172.21.15.66:0/3854572312 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f24b009ad70 con 0x7f24c8102660 2023-12-11T19:32:55.851 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:55.851 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24b40a1590 msgr2=0x7f24b40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24b40a1590 0x7f24b40a3a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24c81024c0 tx=0x7f24b8009260 comp rx=0 tx=0).stop 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 msgr2=0x7f24c819de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c819de70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009990 tx=0x7f24b000fa00 comp rx=0 tx=0).stop 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 shutdown_connections 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24b40a1590 0x7f24b40a3a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24c8101470 0x7f24c819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24c8103000 0x7f24c81a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 --2- 172.21.15.66:0/3854572312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24c8102660 0x7f24c819de70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 >> 172.21.15.66:0/3854572312 conn(0x7f24c80fc9e0 msgr2=0x7f24c8073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 shutdown_connections 2023-12-11T19:32:55.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:55.852+0000 7f24c759e700 1 -- 172.21.15.66:0/3854572312 wait complete. 2023-12-11T19:32:55.855 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:32:55.881 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:32:55.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:55 smithi066 ceph-mon[97920]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:55.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:55 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2379961165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:32:55.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:55 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3015104019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:32:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:55 smithi167 ceph-mon[102750]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:55 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2379961165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:32:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:55 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3015104019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:32:56.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 -- 172.21.15.66:0/3841539593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 msgr2=0x7fc444073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/3841539593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc444073a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc440009990 tx=0x7fc440009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 -- 172.21.15.66:0/3841539593 shutdown_connections 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/3841539593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc44410da70 0x7fc44410fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/3841539593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc444073a60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/3841539593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 0x7fc4440744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 -- 172.21.15.66:0/3841539593 >> 172.21.15.66:0/3841539593 conn(0x7fc4440fca10 msgr2=0x7fc4440fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 -- 172.21.15.66:0/3841539593 shutdown_connections 2023-12-11T19:32:56.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.093+0000 7fc44b9ff700 1 -- 172.21.15.66:0/3841539593 wait complete. 2023-12-11T19:32:56.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.094+0000 7fc44b9ff700 1 Processor -- start 2023-12-11T19:32:56.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 -- start start 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 0x7fc44419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc44419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc44410da70 0x7fc4441a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4441a54b0 con 0x7fc444075610 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4441a55f0 con 0x7fc44410da70 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.095+0000 7fc44b9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4441a5780 con 0x7fc444074030 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc44979b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 0x7fc44419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc44419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc44979b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 0x7fc44419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:58286/0 (socket says 172.21.15.66:58286) 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc44979b700 1 -- 172.21.15.66:0/725180722 learned_addr learned my addr 172.21.15.66:0/725180722 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc449f9c700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc44410da70 0x7fc4441a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 -- 172.21.15.66:0/725180722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 msgr2=0x7fc44419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 0x7fc44419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 -- 172.21.15.66:0/725180722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc44410da70 msgr2=0x7fc4441a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc44410da70 0x7fc4441a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 -- 172.21.15.66:0/725180722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400095f0 con 0x7fc444075610 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc448f9a700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc44419df60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e9c0 tx=0x7fc43400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434004220 con 0x7fc444075610 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc434004380 con 0x7fc444075610 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.096+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434003b40 con 0x7fc444075610 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.097+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4441a5a60 con 0x7fc444075610 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.097+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4441a5f20 con 0x7fc444075610 2023-12-11T19:32:56.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.098+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444103cd0 con 0x7fc444075610 2023-12-11T19:32:56.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.100+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc434003ca0 con 0x7fc444075610 2023-12-11T19:32:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.102+0000 7fc43a7fc700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc4300a1590 0x7fc4300a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.102+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc434014070 con 0x7fc444075610 2023-12-11T19:32:56.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.102+0000 7fc44979b700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc4300a1590 0x7fc4300a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.103+0000 7fc44979b700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc4300a1590 0x7fc4300a3a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc44000b2e0 tx=0x7fc44000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:56.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.106+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc434096530 con 0x7fc444075610 2023-12-11T19:32:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:55 smithi070 ceph-mon[104044]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:55 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2379961165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:32:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:55 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3015104019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:32:56.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.332+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc444113190 con 0x7fc444075610 2023-12-11T19:32:56.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.334+0000 7fc43a7fc700 1 -- 172.21.15.66:0/725180722 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7fc434099b50 con 0x7fc444075610 2023-12-11T19:32:56.335 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:56.336 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.336+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc4300a1590 msgr2=0x7fc4300a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.336+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc4300a1590 0x7fc4300a3a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc44000b2e0 tx=0x7fc44000b610 comp rx=0 tx=0).stop 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.336+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 msgr2=0x7fc44419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.336+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc44419df60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e9c0 tx=0x7fc43400ecd0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 shutdown_connections 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc4300a1590 0x7fc4300a3a50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc44410da70 0x7fc4441a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc444074030 0x7fc44419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 --2- 172.21.15.66:0/725180722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc444075610 0x7fc44419df60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 >> 172.21.15.66:0/725180722 conn(0x7fc4440fca10 msgr2=0x7fc444100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:56.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 shutdown_connections 2023-12-11T19:32:56.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.337+0000 7fc44b9ff700 1 -- 172.21.15.66:0/725180722 wait complete. 2023-12-11T19:32:56.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:32:56.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:32:56.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 -- 172.21.15.66:0/3432108463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 msgr2=0x7f2ffc0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 --2- 172.21.15.66:0/3432108463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc0744b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009990 tx=0x7f2ff0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 -- 172.21.15.66:0/3432108463 shutdown_connections 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 --2- 172.21.15.66:0/3432108463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc10db70 0x7f2ffc10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 --2- 172.21.15.66:0/3432108463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc075610 0x7f2ffc073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 --2- 172.21.15.66:0/3432108463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc0744b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 -- 172.21.15.66:0/3432108463 >> 172.21.15.66:0/3432108463 conn(0x7f2ffc0fcb10 msgr2=0x7f2ffc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 -- 172.21.15.66:0/3432108463 shutdown_connections 2023-12-11T19:32:56.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.579+0000 7f3000fff700 1 -- 172.21.15.66:0/3432108463 wait complete. 2023-12-11T19:32:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.580+0000 7f3000fff700 1 Processor -- start 2023-12-11T19:32:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.580+0000 7f3000fff700 1 -- start start 2023-12-11T19:32:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f3000fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f3000fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc075610 0x7f2ffc19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f3000fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc10db70 0x7f2ffc1a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f3000fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1a5590 con 0x7f2ffc074030 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f3000fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1a56d0 con 0x7f2ffc075610 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f3000fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1a5860 con 0x7f2ffc10db70 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49400/0 (socket says 172.21.15.66:49400) 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 -- 172.21.15.66:0/3611545090 learned_addr learned my addr 172.21.15.66:0/3611545090 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ff9d9b700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc075610 0x7f2ffc19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffad9d700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc10db70 0x7f2ffc1a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 -- 172.21.15.66:0/3611545090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc10db70 msgr2=0x7f2ffc1a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc10db70 0x7f2ffc1a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 -- 172.21.15.66:0/3611545090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc075610 msgr2=0x7f2ffc19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc075610 0x7f2ffc19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ffa59c700 1 -- 172.21.15.66:0/3611545090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00095f0 con 0x7f2ffc074030 2023-12-11T19:32:56.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.581+0000 7f2ff9d9b700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc075610 0x7f2ffc19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.582+0000 7f2ffa59c700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc19dab0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e9a0 tx=0x7f2fec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.582+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec009790 con 0x7f2ffc074030 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.582+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fec010430 con 0x7f2ffc074030 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.582+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc1a5b40 con 0x7f2ffc074030 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.582+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec00cea0 con 0x7f2ffc074030 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.582+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc1a60a0 con 0x7f2ffc074030 2023-12-11T19:32:56.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.583+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc103dd0 con 0x7f2ffc074030 2023-12-11T19:32:56.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.585+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2fec0105a0 con 0x7f2ffc074030 2023-12-11T19:32:56.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.588+0000 7f2feb7fe700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2fe40a1590 0x7f2fe40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:56.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.588+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fec014070 con 0x7f2ffc074030 2023-12-11T19:32:56.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.588+0000 7f2ff9d9b700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2fe40a1590 0x7f2fe40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:56.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.589+0000 7f2ff9d9b700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2fe40a1590 0x7f2fe40a3a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0003f60 tx=0x7f2ff000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:56.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.594+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2fec0522f0 con 0x7f2ffc074030 2023-12-11T19:32:56.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.814+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ffc113240 con 0x7f2ffc074030 2023-12-11T19:32:56.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.815+0000 7f2feb7fe700 1 -- 172.21.15.66:0/3611545090 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2fec0964e0 con 0x7f2ffc074030 2023-12-11T19:32:56.817 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:32:56.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2fe40a1590 msgr2=0x7f2fe40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2fe40a1590 0x7f2fe40a3a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0003f60 tx=0x7f2ff000b610 comp rx=0 tx=0).stop 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 msgr2=0x7f2ffc19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc19dab0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e9a0 tx=0x7f2fec00ecb0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 shutdown_connections 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2fe40a1590 0x7f2fe40a3a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ffc10db70 0x7f2ffc1a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ffc074030 0x7f2ffc19dab0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 --2- 172.21.15.66:0/3611545090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ffc075610 0x7f2ffc19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 >> 172.21.15.66:0/3611545090 conn(0x7f2ffc0fcb10 msgr2=0x7f2ffc100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 shutdown_connections 2023-12-11T19:32:56.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:56.818+0000 7f3000fff700 1 -- 172.21.15.66:0/3611545090 wait complete. 2023-12-11T19:32:56.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:32:57.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:56 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:56 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3854572312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:56 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/725180722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:56 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:56 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3854572312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:56 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/725180722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 -- 172.21.15.66:0/3577496423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c103130 msgr2=0x7f753c1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 --2- 172.21.15.66:0/3577496423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c103130 0x7f753c1099e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7534009fc0 tx=0x7f753400b330 comp rx=0 tx=0).stop 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 -- 172.21.15.66:0/3577496423 shutdown_connections 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 --2- 172.21.15.66:0/3577496423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c103130 0x7f753c1099e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 --2- 172.21.15.66:0/3577496423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c1015a0 0x7f753c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 --2- 172.21.15.66:0/3577496423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.065+0000 7f75445ff700 1 -- 172.21.15.66:0/3577496423 >> 172.21.15.66:0/3577496423 conn(0x7f753c0fcb10 msgr2=0x7f753c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:57.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.066+0000 7f75445ff700 1 -- 172.21.15.66:0/3577496423 shutdown_connections 2023-12-11T19:32:57.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.066+0000 7f75445ff700 1 -- 172.21.15.66:0/3577496423 wait complete. 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 Processor -- start 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 -- start start 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c1015a0 0x7f753c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c103130 0x7f753c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1a12b0 con 0x7f753c102790 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1a13f0 con 0x7f753c1015a0 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f75445ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1a1580 con 0x7f753c103130 2023-12-11T19:32:57.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f7541b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f7541b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49412/0 (socket says 172.21.15.66:49412) 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f7541b9a700 1 -- 172.21.15.66:0/2331475564 learned_addr learned my addr 172.21.15.66:0/2331475564 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f754239b700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c1015a0 0x7f753c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.067+0000 7f7542b9c700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c103130 0x7f753c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f7541b9a700 1 -- 172.21.15.66:0/2331475564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c103130 msgr2=0x7f753c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f7541b9a700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c103130 0x7f753c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f7541b9a700 1 -- 172.21.15.66:0/2331475564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c1015a0 msgr2=0x7f753c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f7541b9a700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c1015a0 0x7f753c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f7541b9a700 1 -- 172.21.15.66:0/2331475564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534009c40 con 0x7f753c102790 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f754239b700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c1015a0 0x7f753c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f7541b9a700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c199cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e9c0 tx=0x7f752c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c004a30 con 0x7f753c102790 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f752c00cc50 con 0x7f753c102790 2023-12-11T19:32:57.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c1a1860 con 0x7f753c102790 2023-12-11T19:32:57.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c005410 con 0x7f753c102790 2023-12-11T19:32:57.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.068+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c1a1dc0 con 0x7f753c102790 2023-12-11T19:32:57.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.069+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c068310 con 0x7f753c102790 2023-12-11T19:32:57.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.072+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f752c005570 con 0x7f753c102790 2023-12-11T19:32:57.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.073+0000 7f75337fe700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75280a1590 0x7f75280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.073+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f752c014070 con 0x7f753c102790 2023-12-11T19:32:57.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.073+0000 7f754239b700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75280a1590 0x7f75280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.074+0000 7f754239b700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75280a1590 0x7f75280a3a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f753c1025f0 tx=0x7f753800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:57.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.075+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f752c096630 con 0x7f753c102790 2023-12-11T19:32:57.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:56 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:56 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3854572312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:56 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/725180722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:57.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.296+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f753c10ce50 con 0x7f753c102790 2023-12-11T19:32:57.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.297+0000 7f75337fe700 1 -- 172.21.15.66:0/2331475564 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f753c10ce50 con 0x7f753c102790 2023-12-11T19:32:57.298 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75280a1590 msgr2=0x7f75280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75280a1590 0x7f75280a3a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f753c1025f0 tx=0x7f753800b410 comp rx=0 tx=0).stop 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 msgr2=0x7f753c199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c199cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e9c0 tx=0x7f752c00ecd0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 shutdown_connections 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f75280a1590 0x7f75280a3a50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f753c1015a0 0x7f753c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f753c103130 0x7f753c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 --2- 172.21.15.66:0/2331475564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f753c102790 0x7f753c199cc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.299+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 >> 172.21.15.66:0/2331475564 conn(0x7f753c0fcb10 msgr2=0x7f753c107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.300+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 shutdown_connections 2023-12-11T19:32:57.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.300+0000 7f75445ff700 1 -- 172.21.15.66:0/2331475564 wait complete. 2023-12-11T19:32:57.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:32:57.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 -- 172.21.15.66:0/524846915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 msgr2=0x7f97a00744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/524846915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a00744b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9794009990 tx=0x7f9794009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 -- 172.21.15.66:0/524846915 shutdown_connections 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/524846915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a010d9b0 0x7f97a010fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/524846915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a0075610 0x7f97a0073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/524846915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a00744b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 -- 172.21.15.66:0/524846915 >> 172.21.15.66:0/524846915 conn(0x7f97a00fc930 msgr2=0x7f97a00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 -- 172.21.15.66:0/524846915 shutdown_connections 2023-12-11T19:32:57.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.543+0000 7f97a4f9d700 1 -- 172.21.15.66:0/524846915 wait complete. 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 Processor -- start 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 -- start start 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a0075610 0x7f97a019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a010d9b0 0x7f97a01a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a01a5490 con 0x7f97a0074030 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a01a55d0 con 0x7f97a0075610 2023-12-11T19:32:57.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.544+0000 7f97a4f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a01a5760 con 0x7f97a010d9b0 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49432/0 (socket says 172.21.15.66:49432) 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 -- 172.21.15.66:0/1097785747 learned_addr learned my addr 172.21.15.66:0/1097785747 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979ffff700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a010d9b0 0x7f97a01a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979effd700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a0075610 0x7f97a019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 -- 172.21.15.66:0/1097785747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a010d9b0 msgr2=0x7f97a01a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a010d9b0 0x7f97a01a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 -- 172.21.15.66:0/1097785747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a0075610 msgr2=0x7f97a019def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a0075610 0x7f97a019def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 -- 172.21.15.66:0/1097785747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940095f0 con 0x7f97a0074030 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.545+0000 7f979f7fe700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a019d9b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9a0 tx=0x7f979000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.546+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790009790 con 0x7f97a0074030 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.546+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a01a5a40 con 0x7f97a0074030 2023-12-11T19:32:57.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.546+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9790010430 con 0x7f97a0074030 2023-12-11T19:32:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.546+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000cea0 con 0x7f97a0074030 2023-12-11T19:32:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.546+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a01a5f50 con 0x7f97a0074030 2023-12-11T19:32:57.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.547+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a0103c10 con 0x7f97a0074030 2023-12-11T19:32:57.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.549+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f97900105a0 con 0x7f97a0074030 2023-12-11T19:32:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.551+0000 7f979cff9700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f97880a1510 0x7f97880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.551+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9790014070 con 0x7f97a0074030 2023-12-11T19:32:57.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.551+0000 7f979effd700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f97880a1510 0x7f97880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:57.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.552+0000 7f979effd700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f97880a1510 0x7f97880a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f979400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:57.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.555+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f97900522f0 con 0x7f97a0074030 2023-12-11T19:32:57.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.777+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97a01130d0 con 0x7f97a0074030 2023-12-11T19:32:57.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.777+0000 7f979cff9700 1 -- 172.21.15.66:0/1097785747 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9790096550 con 0x7f97a0074030 2023-12-11T19:32:57.779 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:32:57.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f97880a1510 msgr2=0x7f97880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f97880a1510 0x7f97880a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f979400b610 comp rx=0 tx=0).stop 2023-12-11T19:32:57.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 msgr2=0x7f97a019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a019d9b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9a0 tx=0x7f979000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 shutdown_connections 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f97880a1510 0x7f97880a39d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97a010d9b0 0x7f97a01a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f97a0074030 0x7f97a019d9b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 --2- 172.21.15.66:0/1097785747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97a0075610 0x7f97a019def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 >> 172.21.15.66:0/1097785747 conn(0x7f97a00fc930 msgr2=0x7f97a0100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 shutdown_connections 2023-12-11T19:32:57.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:57.780+0000 7f97a4f9d700 1 -- 172.21.15.66:0/1097785747 wait complete. 2023-12-11T19:32:57.803 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:32:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.017+0000 7f2587fff700 1 -- 172.21.15.66:0/3226180913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25801036b0 msgr2=0x7f2580103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.017+0000 7f2587fff700 1 --2- 172.21.15.66:0/3226180913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25801036b0 0x7f2580103ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f257c0099c0 tx=0x7f257c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:57 smithi167 ceph-mon[102750]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:32:58.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:57 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3611545090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:32:58.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:57 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2331475564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 -- 172.21.15.66:0/3226180913 shutdown_connections 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 --2- 172.21.15.66:0/3226180913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25801036b0 0x7f2580103ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 --2- 172.21.15.66:0/3226180913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2580105240 0x7f258010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 --2- 172.21.15.66:0/3226180913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801048a0 0x7f2580104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 -- 172.21.15.66:0/3226180913 >> 172.21.15.66:0/3226180913 conn(0x7f25800fec40 msgr2=0x7f2580101060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 -- 172.21.15.66:0/3226180913 shutdown_connections 2023-12-11T19:32:58.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.019+0000 7f2587fff700 1 -- 172.21.15.66:0/3226180913 wait complete. 2023-12-11T19:32:58.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.020+0000 7f2587fff700 1 Processor -- start 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.020+0000 7f2587fff700 1 -- start start 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.020+0000 7f2587fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 0x7f2580199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.020+0000 7f2587fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25801048a0 0x7f2580199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.020+0000 7f2587fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580105240 0x7f25801a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2587fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a10a0 con 0x7f25801048a0 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2587fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a11e0 con 0x7f2580105240 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2587fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a1370 con 0x7f25801036b0 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 0x7f2580199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f258559a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25801048a0 0x7f2580199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 0x7f2580199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:58358/0 (socket says 172.21.15.66:58358) 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 -- 172.21.15.66:0/2442003111 learned_addr learned my addr 172.21.15.66:0/2442003111 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f258659c700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580105240 0x7f25801a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 -- 172.21.15.66:0/2442003111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580105240 msgr2=0x7f25801a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580105240 0x7f25801a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 -- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25801048a0 msgr2=0x7f2580199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25801048a0 0x7f2580199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.021+0000 7f2585d9b700 1 -- 172.21.15.66:0/2442003111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0095f0 con 0x7f25801036b0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.022+0000 7f2585d9b700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 0x7f2580199570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f257c000c00 tx=0x7f257c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.022+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c01d070 con 0x7f25801036b0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.022+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f257c005050 con 0x7f25801036b0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.022+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25801a15f0 con 0x7f25801036b0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.022+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c004a70 con 0x7f25801036b0 2023-12-11T19:32:58.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.022+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25801a1a10 con 0x7f25801036b0 2023-12-11T19:32:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.023+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258010abd0 con 0x7f25801036b0 2023-12-11T19:32:58.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.025+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f257c0051c0 con 0x7f25801036b0 2023-12-11T19:32:58.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.026+0000 7f2576ffd700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f256c0a1510 0x7f256c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.026+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f257c0cdbd0 con 0x7f25801036b0 2023-12-11T19:32:58.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.027+0000 7f258559a700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f256c0a1510 0x7f256c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.027+0000 7f258559a700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f256c0a1510 0x7f256c0a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2570009c60 tx=0x7f2570009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:58.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.031+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f257c097120 con 0x7f25801036b0 2023-12-11T19:32:58.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:57 smithi066 ceph-mon[97920]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:32:58.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:57 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3611545090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:32:58.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:57 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2331475564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:32:58.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:57 smithi070 ceph-mon[104044]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:32:58.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:57 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3611545090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:32:58.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:57 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2331475564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:32:58.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.300+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f258010eff0 con 0x7f25801036b0 2023-12-11T19:32:58.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.302+0000 7f2576ffd700 1 -- 172.21.15.66:0/2442003111 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f257c09a740 con 0x7f25801036b0 2023-12-11T19:32:58.303 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:58.303 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f256c0a1510 msgr2=0x7f256c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f256c0a1510 0x7f256c0a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2570009c60 tx=0x7f2570009260 comp rx=0 tx=0).stop 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 msgr2=0x7f2580199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 0x7f2580199570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f257c000c00 tx=0x7f257c00f690 comp rx=0 tx=0).stop 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 shutdown_connections 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f256c0a1510 0x7f256c0a39d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580105240 0x7f25801a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25801036b0 0x7f2580199570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 --2- 172.21.15.66:0/2442003111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25801048a0 0x7f2580199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 >> 172.21.15.66:0/2442003111 conn(0x7f25800fec40 msgr2=0x7f2580107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 shutdown_connections 2023-12-11T19:32:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.304+0000 7f2587fff700 1 -- 172.21.15.66:0/2442003111 wait complete. 2023-12-11T19:32:58.307 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:32:58.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 -- 172.21.15.66:0/2212319897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d80102690 msgr2=0x7f9d80102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 --2- 172.21.15.66:0/2212319897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d80102690 0x7f9d80102af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009990 tx=0x7f9d74009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 -- 172.21.15.66:0/2212319897 shutdown_connections 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 --2- 172.21.15.66:0/2212319897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d801014a0 0x7f9d801018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 --2- 172.21.15.66:0/2212319897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80103030 0x7f9d801098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 --2- 172.21.15.66:0/2212319897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d80102690 0x7f9d80102af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 -- 172.21.15.66:0/2212319897 >> 172.21.15.66:0/2212319897 conn(0x7f9d800fca10 msgr2=0x7f9d800fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 -- 172.21.15.66:0/2212319897 shutdown_connections 2023-12-11T19:32:58.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.539+0000 7f9d85bff700 1 -- 172.21.15.66:0/2212319897 wait complete. 2023-12-11T19:32:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 Processor -- start 2023-12-11T19:32:58.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 -- start start 2023-12-11T19:32:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d801014a0 0x7f9d80197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 0x7f9d80197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d80103030 0x7f9d8019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8019ee60 con 0x7f9d80102690 2023-12-11T19:32:58.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8019efa0 con 0x7f9d80103030 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.541+0000 7f9d85bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8019f130 con 0x7f9d801014a0 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 0x7f9d80197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 0x7f9d80197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49468/0 (socket says 172.21.15.66:49468) 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 -- 172.21.15.66:0/557291720 learned_addr learned my addr 172.21.15.66:0/557291720 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7ffff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d80103030 0x7f9d8019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7f7fe700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d801014a0 0x7f9d80197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 -- 172.21.15.66:0/557291720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d801014a0 msgr2=0x7f9d80197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d801014a0 0x7f9d80197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 -- 172.21.15.66:0/557291720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d80103030 msgr2=0x7f9d8019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d80103030 0x7f9d8019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 -- 172.21.15.66:0/557291720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740095f0 con 0x7f9d80102690 2023-12-11T19:32:58.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7ffff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d80103030 0x7f9d8019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.542+0000 7f9d7effd700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 0x7f9d80197870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005910 tx=0x7f9d7400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.543+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7401d070 con 0x7f9d80102690 2023-12-11T19:32:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.543+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d7400bed0 con 0x7f9d80102690 2023-12-11T19:32:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.543+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400fdc0 con 0x7f9d80102690 2023-12-11T19:32:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.543+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8019f3b0 con 0x7f9d80102690 2023-12-11T19:32:58.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.543+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8019f780 con 0x7f9d80102690 2023-12-11T19:32:58.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.544+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d801089c0 con 0x7f9d80102690 2023-12-11T19:32:58.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.546+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9d7400f560 con 0x7f9d80102690 2023-12-11T19:32:58.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.547+0000 7f9d7cff9700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9d680a1510 0x7f9d680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:58.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.548+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9d740ce530 con 0x7f9d80102690 2023-12-11T19:32:58.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.548+0000 7f9d7f7fe700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9d680a1510 0x7f9d680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:58.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.548+0000 7f9d7f7fe700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9d680a1510 0x7f9d680a39d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d801024f0 tx=0x7f9d70009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.553+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9d74097a80 con 0x7f9d80102690 2023-12-11T19:32:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.783+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d8010cbc0 con 0x7f9d80102690 2023-12-11T19:32:58.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.784+0000 7f9d7cff9700 1 -- 172.21.15.66:0/557291720 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f9d7409b0a0 con 0x7f9d80102690 2023-12-11T19:32:58.786 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:58.786 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:32:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9d680a1510 msgr2=0x7f9d680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9d680a1510 0x7f9d680a39d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d801024f0 tx=0x7f9d70009260 comp rx=0 tx=0).stop 2023-12-11T19:32:58.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 msgr2=0x7f9d80197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 0x7f9d80197870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005910 tx=0x7f9d7400b9c0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 shutdown_connections 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9d680a1510 0x7f9d680a39d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d80103030 0x7f9d8019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d801014a0 0x7f9d80197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 --2- 172.21.15.66:0/557291720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d80102690 0x7f9d80197870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 >> 172.21.15.66:0/557291720 conn(0x7f9d800fca10 msgr2=0x7f9d80105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 shutdown_connections 2023-12-11T19:32:58.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:58.787+0000 7f9d85bff700 1 -- 172.21.15.66:0/557291720 wait complete. 2023-12-11T19:32:58.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:32:59.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:58 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1097785747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:32:59.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:58 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2442003111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 -- 172.21.15.66:0/2615634477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248073fd0 msgr2=0x7f1248074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 --2- 172.21.15.66:0/2615634477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248073fd0 0x7f1248074450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f123c009990 tx=0x7f123c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 -- 172.21.15.66:0/2615634477 shutdown_connections 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 --2- 172.21.15.66:0/2615634477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248109380 0x7f124810b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 --2- 172.21.15.66:0/2615634477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12480755b0 0x7f1248073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 --2- 172.21.15.66:0/2615634477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1248073fd0 0x7f1248074450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 -- 172.21.15.66:0/2615634477 >> 172.21.15.66:0/2615634477 conn(0x7f12480fca20 msgr2=0x7f12480fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 -- 172.21.15.66:0/2615634477 shutdown_connections 2023-12-11T19:32:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.023+0000 7f124dbff700 1 -- 172.21.15.66:0/2615634477 wait complete. 2023-12-11T19:32:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.024+0000 7f124dbff700 1 Processor -- start 2023-12-11T19:32:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 -- start start 2023-12-11T19:32:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248073fd0 0x7f1248199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12480755b0 0x7f1248199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 0x7f12481a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12481a10a0 con 0x7f1248109380 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12481a11e0 con 0x7f1248073fd0 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f124dbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12481a1370 con 0x7f12480755b0 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f1247fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 0x7f12481a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f12477fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248073fd0 0x7f1248199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f1247fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 0x7f12481a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49474/0 (socket says 172.21.15.66:49474) 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f1247fff700 1 -- 172.21.15.66:0/2538259120 learned_addr learned my addr 172.21.15.66:0/2538259120 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.025+0000 7f1246ffd700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12480755b0 0x7f1248199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1247fff700 1 -- 172.21.15.66:0/2538259120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12480755b0 msgr2=0x7f1248199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1247fff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12480755b0 0x7f1248199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1247fff700 1 -- 172.21.15.66:0/2538259120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248073fd0 msgr2=0x7f1248199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1247fff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248073fd0 0x7f1248199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1247fff700 1 -- 172.21.15.66:0/2538259120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009c40 con 0x7f1248109380 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f12477fe700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248073fd0 0x7f1248199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1247fff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 0x7f12481a0ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f124000c370 tx=0x7f124000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240017070 con 0x7f1248109380 2023-12-11T19:32:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c0095f0 con 0x7f1248109380 2023-12-11T19:32:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f124000f040 con 0x7f1248109380 2023-12-11T19:32:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124000e050 con 0x7f1248109380 2023-12-11T19:32:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.026+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12481a1970 con 0x7f1248109380 2023-12-11T19:32:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.027+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1248108190 con 0x7f1248109380 2023-12-11T19:32:59.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.030+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f124000cb60 con 0x7f1248109380 2023-12-11T19:32:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.032+0000 7f1244ff9700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f12300a1510 0x7f12300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.032+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f12400ccec0 con 0x7f1248109380 2023-12-11T19:32:59.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.032+0000 7f12477fe700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f12300a1510 0x7f12300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.033+0000 7f12477fe700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f12300a1510 0x7f12300a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1238011490 tx=0x7f1238009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.034+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1240096380 con 0x7f1248109380 2023-12-11T19:32:59.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:58 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1097785747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:32:59.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:58 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2442003111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:59.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:58 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1097785747' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:32:59.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:58 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2442003111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:59.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.308+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1248068230 con 0x7f1248109380 2023-12-11T19:32:59.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.309+0000 7f1244ff9700 1 -- 172.21.15.66:0/2538259120 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f12400999a0 con 0x7f1248109380 2023-12-11T19:32:59.311 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:32:59.311 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f12300a1510 msgr2=0x7f12300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f12300a1510 0x7f12300a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1238011490 tx=0x7f1238009260 comp rx=0 tx=0).stop 2023-12-11T19:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 msgr2=0x7f12481a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 0x7f12481a0ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f124000c370 tx=0x7f124000c680 comp rx=0 tx=0).stop 2023-12-11T19:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 shutdown_connections 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f12300a1510 0x7f12300a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248073fd0 0x7f1248199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1248109380 0x7f12481a0ad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 --2- 172.21.15.66:0/2538259120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12480755b0 0x7f1248199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 >> 172.21.15.66:0/2538259120 conn(0x7f12480fca20 msgr2=0x7f12481052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 shutdown_connections 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.312+0000 7f124dbff700 1 -- 172.21.15.66:0/2538259120 wait complete. 2023-12-11T19:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:32:59.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-11T19:31:51.764249+0000', 'modified': '2023-12-11T19:31:59.541695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24575, 'mds_1': 24547, 'mds_2': 24514, 'mds_3': 24494, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14673': {'gid': 14673, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6839/2148751938', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 2148751938}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 2148751938}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14685': {'gid': 14685, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6841/1874182059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6840', 'nonce': 1874182059}, {'type': 'v1', 'addr': '172.21.15.66:6841', 'nonce': 1874182059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/1537942709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 1537942709}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 1537942709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24467': {'gid': 24467, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.70:6833/1581467356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 1581467356}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 1581467356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.70:6835/3670150002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 3670150002}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 3670150002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.167:6835/1867143564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1867143564}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 1867143564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2775450491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2775450491}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2775450491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.70:6837/280227891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 280227891}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 280227891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.167:6839/3906090427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3906090427}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3906090427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6839/725803001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6838', 'nonce': 725803001}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 725803001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-11T19:32:59.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-11T19:32:59.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-12-11T19:32:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 -- 172.21.15.66:0/683709203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67381015a0 msgr2=0x7f67381019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 --2- 172.21.15.66:0/683709203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67381015a0 0x7f67381019c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67280099c0 tx=0x7f6728009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 -- 172.21.15.66:0/683709203 shutdown_connections 2023-12-11T19:32:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 --2- 172.21.15.66:0/683709203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67381015a0 0x7f67381019c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 --2- 172.21.15.66:0/683709203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738103130 0x7f67381099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 --2- 172.21.15.66:0/683709203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6738102790 0x7f6738102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 -- 172.21.15.66:0/683709203 >> 172.21.15.66:0/683709203 conn(0x7f67380fcb10 msgr2=0x7f67380fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.552+0000 7f673cfff700 1 -- 172.21.15.66:0/683709203 shutdown_connections 2023-12-11T19:32:59.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.553+0000 7f673cfff700 1 -- 172.21.15.66:0/683709203 wait complete. 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.553+0000 7f673cfff700 1 Processor -- start 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 -- start start 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67381015a0 0x7f6738199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 0x7f6738199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738103130 0x7f67381a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67381a1230 con 0x7f6738102790 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67381a1370 con 0x7f6738103130 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673cfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67381a1500 con 0x7f67381015a0 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f672ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 0x7f6738199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67381015a0 0x7f6738199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f6736d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738103130 0x7f67381a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f672ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 0x7f6738199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49490/0 (socket says 172.21.15.66:49490) 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f673659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67381015a0 0x7f6738199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:58420/0 (socket says 172.21.15.66:58420) 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.554+0000 7f672ffff700 1 -- 172.21.15.66:0/1712951487 learned_addr learned my addr 172.21.15.66:0/1712951487 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672ffff700 1 -- 172.21.15.66:0/1712951487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67381015a0 msgr2=0x7f6738199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672ffff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67381015a0 0x7f6738199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672ffff700 1 -- 172.21.15.66:0/1712951487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738103130 msgr2=0x7f67381a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672ffff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738103130 0x7f67381a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672ffff700 1 -- 172.21.15.66:0/1712951487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280095f0 con 0x7f6738102790 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f6736d9d700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738103130 0x7f67381a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672ffff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 0x7f6738199c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f672000e9a0 tx=0x7f672000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672f7fe700 1 -- 172.21.15.66:0/1712951487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720009790 con 0x7f6738102790 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67381a17e0 con 0x7f6738102790 2023-12-11T19:32:59.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672f7fe700 1 -- 172.21.15.66:0/1712951487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6720010430 con 0x7f6738102790 2023-12-11T19:32:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f672f7fe700 1 -- 172.21.15.66:0/1712951487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672000cea0 con 0x7f6738102790 2023-12-11T19:32:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.555+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67381a1cf0 con 0x7f6738102790 2023-12-11T19:32:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.558+0000 7f672f7fe700 1 -- 172.21.15.66:0/1712951487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f67200105a0 con 0x7f6738102790 2023-12-11T19:32:59.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 Processor -- start 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 -- start start 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f672f7fe700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f67240a1510 0x7f67240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 0x563083c017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f672f7fe700 1 -- 172.21.15.66:0/1712951487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6720014070 con 0x7f6738102790 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 0x563083c025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083b5cb00 con 0x563083c02bd0 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083b5a420 con 0x563083c01ce0 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.561+0000 7f6750866b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083b55d10 con 0x563083c01380 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f6736d9d700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f67240a1510 0x7f67240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 0x563083c025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f673659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 0x563083c017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 0x563083c025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52176/0 (socket says 172.21.15.66:52176) 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f673659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 0x563083c017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:58434/0 (socket says 172.21.15.66:58434) 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 -- 172.21.15.66:0/446950258 learned_addr learned my addr 172.21.15.66:0/446950258 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f6736d9d700 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 -- 172.21.15.66:0/446950258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 msgr2=0x563083c017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 0x563083c017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 -- 172.21.15.66:0/446950258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 msgr2=0x563083c03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c03050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 -- 172.21.15.66:0/446950258 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630839d25e0 con 0x563083c01ce0 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f673659c700 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 0x563083c017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f6736d9d700 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:59.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672ffff700 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 0x563083c025f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6720000c00 tx=0x7f6720099e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9689bebc009f3a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f6736d9d700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f67240a1510 0x7f67240a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x563083c023b0 tx=0x7f6730009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.562+0000 7f672cff9700 1 -- 172.21.15.66:0/446950258 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672009e120 con 0x563083c01ce0 2023-12-11T19:32:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f672cff9700 1 -- 172.21.15.66:0/446950258 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672009a210 con 0x563083c01ce0 2023-12-11T19:32:59.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f672cff9700 1 -- 172.21.15.66:0/446950258 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720099960 con 0x563083c01ce0 2023-12-11T19:32:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 -- 172.21.15.66:0/446950258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 msgr2=0x563083c025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 0x563083c025f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6720000c00 tx=0x7f6720099e80 comp rx=0 tx=0).stop 2023-12-11T19:32:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 -- 172.21.15.66:0/446950258 shutdown_connections 2023-12-11T19:32:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01380 0x563083c017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 --2- 172.21.15.66:0/446950258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01ce0 0x563083c025f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 -- 172.21.15.66:0/446950258 >> 172.21.15.66:0/446950258 conn(0x563083b30790 msgr2=0x563083bfdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 -- 172.21.15.66:0/446950258 shutdown_connections 2023-12-11T19:32:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.563+0000 7f6750866b80 1 -- 172.21.15.66:0/446950258 wait complete. 2023-12-11T19:32:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.564+0000 7f6750866b80 1 -- start start 2023-12-11T19:32:59.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.564+0000 7f6750866b80 1 Processor -- start 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 0x563083c0b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c12770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083b0f510 con 0x563083c02bd0 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083b0f1b0 con 0x563083c01380 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630839d48d0 con 0x563083c01ce0 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6750866b80 10 client.0 _ll_register_callbacks cb 0x563083a3aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:32:59.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 0x563083c0b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 0x563083c0b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52190/0 (socket says 172.21.15.66:52190) 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 -- 172.21.15.66:0/2469137639 learned_addr learned my addr 172.21.15.66:0/2469137639 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f6736d9d700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c12770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f672ffff700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 -- 172.21.15.66:0/2469137639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 msgr2=0x563083c0b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 -- 172.21.15.66:0/2469137639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 msgr2=0x563083c12770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c12770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.565+0000 7f673659c700 1 -- 172.21.15.66:0/2469137639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673810a330 con 0x563083c01380 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f673659c700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 0x563083c0b1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6728000c00 tx=0x7f6728005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=331f78f161fd420c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f671dffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f671dffb700 1 -- 172.21.15.66:0/2469137639 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800bd40 con 0x563083c01380 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563083bf4af0 con 0x563083c01380 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f671dffb700 1 -- 172.21.15.66:0/2469137639 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6728003760 con 0x563083c01380 2023-12-11T19:32:59.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f671dffb700 1 -- 172.21.15.66:0/2469137639 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6728003920 con 0x563083c01380 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f6750866b80 10 client.24932 fetch_fsmap learned FSMap version 27 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f67381a1cf0 con 0x563083c01380 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.566+0000 7f671dffb700 1 -- 172.21.15.66:0/2469137639 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563083bf4af0 con 0x563083c01380 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f671dffb700 1 -- 172.21.15.66:0/2469137639 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6728024a90 con 0x563083c01380 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f6750866b80 10 client.24932 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f6750866b80 10 client.24932 resolve_mds: resolved name 'k' to daemon mds.k 2023-12-11T19:32:59.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f6750866b80 20 client.24932 populate_metadata read hostname 'smithi066' 2023-12-11T19:32:59.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f6750866b80 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c16e10 0x563083c15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f6750866b80 4 client.24932 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:32:59.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563083bd0ef0 con 0x563083c16e10 2023-12-11T19:32:59.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.567+0000 7f672ffff700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c16e10 0x563083c15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.568+0000 7f671dffb700 10 client.24932 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.568+0000 7f672ffff700 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c16e10 0x563083c15910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5235abeb3ab6354 server_cookie=10aacdc438ace50 in_seq=0 out_seq=0 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f671dffb700 1 -- 172.21.15.66:0/2469137639 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563083bd0ef0 con 0x563083c16e10 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f671dffb700 10 client.24932 handle_command_reply: tid=0 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 client.24932 shutdown 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 msgr2=0x563083c0b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 0x563083c0b1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6728000c00 tx=0x7f6728005270 comp rx=0 tx=0).stop 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 shutdown_connections 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c16e10 0x563083c15910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c01380 0x563083c0b1f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c02bd0 0x563083c12770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f671dffb700 0 client.24932 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 --2- 172.21.15.66:0/2469137639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 >> 172.21.15.66:0/2469137639 conn(0x563083b30790 msgr2=0x563083bfc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 shutdown_connections 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.569+0000 7f6750866b80 1 -- 172.21.15.66:0/2469137639 wait complete. 2023-12-11T19:32:59.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.570+0000 7f6750866b80 20 client.24932 trim_cache size 0 max 16384 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 Processor -- start 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 -- start start 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083bfd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c12730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083bfde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083af1360 con 0x563083c01380 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083af8830 con 0x563083c02bd0 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f6750866b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083c0ad80 con 0x563083c01ce0 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.589+0000 7f673659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083bfd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f6736d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083bfde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083bfd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49518/0 (socket says 172.21.15.66:49518) 2023-12-11T19:32:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 -- 172.21.15.66:0/2133874349 learned_addr learned my addr 172.21.15.66:0/2133874349 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f672ffff700 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c12730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 -- 172.21.15.66:0/2133874349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 msgr2=0x563083c12730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c12730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 -- 172.21.15.66:0/2133874349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 msgr2=0x563083bfde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083bfde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 -- 172.21.15.66:0/2133874349 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56308397ed90 con 0x563083c01380 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f6736d9d700 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083bfde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f673659c700 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083bfd920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6728001e90 tx=0x7f6728009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4759dbf39bb0c18 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.590+0000 7f672d7fa700 1 -- 172.21.15.66:0/2133874349 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800bd40 con 0x563083c01380 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f672d7fa700 1 -- 172.21.15.66:0/2133874349 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672801e040 con 0x563083c01380 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f672d7fa700 1 -- 172.21.15.66:0/2133874349 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728003ae0 con 0x563083c01380 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 -- 172.21.15.66:0/2133874349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 msgr2=0x563083bfd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083bfd920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6728001e90 tx=0x7f6728009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 -- 172.21.15.66:0/2133874349 shutdown_connections 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083bfde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083bfd920 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 --2- 172.21.15.66:0/2133874349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c12730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 -- 172.21.15.66:0/2133874349 >> 172.21.15.66:0/2133874349 conn(0x563083b30790 msgr2=0x563083c05b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 -- 172.21.15.66:0/2133874349 shutdown_connections 2023-12-11T19:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.591+0000 7f6750866b80 1 -- 172.21.15.66:0/2133874349 wait complete. 2023-12-11T19:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.592+0000 7f6750866b80 1 -- start start 2023-12-11T19:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.592+0000 7f6750866b80 1 Processor -- start 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083c12400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083c0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083c0fa70 con 0x563083c01380 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083c0fbe0 con 0x563083c02bd0 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563083c6cf00 con 0x563083c01ce0 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6750866b80 10 client.0 _ll_register_callbacks cb 0x563083a3aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f673659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083c12400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f672ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f672ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:58454/0 (socket says 172.21.15.66:58454) 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f672ffff700 1 -- 172.21.15.66:0/369475379 learned_addr learned my addr 172.21.15.66:0/369475379 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.593+0000 7f6736d9d700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083c0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f672ffff700 1 -- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 msgr2=0x563083c0f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f672ffff700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083c0f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f672ffff700 1 -- 172.21.15.66:0/369475379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 msgr2=0x563083c12400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f672ffff700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083c12400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f672ffff700 1 -- 172.21.15.66:0/369475379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6730008df0 con 0x563083c01ce0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f673659c700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083c12400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f672ffff700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0efc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6720099f10 tx=0x7f6720099fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bf6e1708acf73a server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f671dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f671dffb700 1 -- 172.21.15.66:0/369475379 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720099900 con 0x563083c01ce0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f672009a210 con 0x563083c01ce0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f671dffb700 1 -- 172.21.15.66:0/369475379 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f672009e360 con 0x563083c01ce0 2023-12-11T19:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.594+0000 7f671dffb700 1 -- 172.21.15.66:0/369475379 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67200a25a0 con 0x563083c01ce0 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 10 client.24925 fetch_fsmap learned FSMap version 27 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f67381a1cf0 con 0x563083c01ce0 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f671dffb700 1 -- 172.21.15.66:0/369475379 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672009a210 con 0x563083c01ce0 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f671dffb700 1 -- 172.21.15.66:0/369475379 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f67200a2110 con 0x563083c01ce0 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 10 client.24925 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 10 client.24925 resolve_mds: resolved name 'k' to daemon mds.k 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 20 client.24925 populate_metadata read hostname 'smithi066' 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c71f60 0x563083c6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 4 client.24925 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563083b1bf80 con 0x563083c71f60 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.595+0000 7f673659c700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c71f60 0x563083c6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f673659c700 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c71f60 0x563083c6e3c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cecfd77123567e4a server_cookie=d099ad3270104e8f in_seq=0 out_seq=0 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f671dffb700 10 client.24925 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f671dffb700 1 -- 172.21.15.66:0/369475379 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563083b1bf80 con 0x563083c71f60 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f671dffb700 10 client.24925 handle_command_reply: tid=0 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 client.24925 shutdown 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 msgr2=0x563083c0efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0efc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6720099f10 tx=0x7f6720099fe0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 shutdown_connections 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563083c71f60 0x563083c6e3c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563083c02bd0 0x563083c0f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563083c01380 0x563083c12400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 --2- 172.21.15.66:0/369475379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563083c01ce0 0x563083c0efc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f671dffb700 0 client.24925 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.596+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 >> 172.21.15.66:0/369475379 conn(0x563083b30790 msgr2=0x563083bfd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 shutdown_connections 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f6750866b80 1 -- 172.21.15.66:0/369475379 wait complete. 2023-12-11T19:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f6750866b80 20 client.24925 trim_cache size 0 max 16384 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 4, 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24467, 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:32:59.602 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 61.059287445999999, 2023-12-11T19:32:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:32:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:32:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:32:59.603 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 105.744871222 2023-12-11T19:32:59.603 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f67240a1510 msgr2=0x7f67240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f673cfff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f67240a1510 0x7f67240a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x563083c023b0 tx=0x7f6730009920 comp rx=0 tx=0).stop 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 msgr2=0x7f6738199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.597+0000 7f673cfff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 0x7f6738199c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f672000e9a0 tx=0x7f672000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 shutdown_connections 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f67240a1510 0x7f67240a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738103130 0x7f67381a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67381015a0 0x7f6738199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 --2- 172.21.15.66:0/1712951487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6738102790 0x7f6738199c90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 >> 172.21.15.66:0/1712951487 conn(0x7f67380fcb10 msgr2=0x7f6738107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 shutdown_connections 2023-12-11T19:32:59.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.598+0000 7f673cfff700 1 -- 172.21.15.66:0/1712951487 wait complete. 2023-12-11T19:32:59.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-12-11T19:32:59.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.833+0000 7f62d49ff700 1 -- 172.21.15.66:0/3301764608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 msgr2=0x7f62d01019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.833+0000 7f62d49ff700 1 --2- 172.21.15.66:0/3301764608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d01019c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f62c00099c0 tx=0x7f62c0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 -- 172.21.15.66:0/3301764608 shutdown_connections 2023-12-11T19:32:59.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 --2- 172.21.15.66:0/3301764608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0103130 0x7f62d01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 --2- 172.21.15.66:0/3301764608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d01019c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 --2- 172.21.15.66:0/3301764608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0102790 0x7f62d0102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 -- 172.21.15.66:0/3301764608 >> 172.21.15.66:0/3301764608 conn(0x7f62d00fcb10 msgr2=0x7f62d00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 -- 172.21.15.66:0/3301764608 shutdown_connections 2023-12-11T19:32:59.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.834+0000 7f62d49ff700 1 -- 172.21.15.66:0/3301764608 wait complete. 2023-12-11T19:32:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 Processor -- start 2023-12-11T19:32:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 -- start start 2023-12-11T19:32:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0102790 0x7f62d0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0103130 0x7f62d01a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01a1230 con 0x7f62d01015a0 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01a1370 con 0x7f62d0102790 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62d49ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01a1500 con 0x7f62d0103130 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62ce59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62cdd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0102790 0x7f62d0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62cdd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0102790 0x7f62d0199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52218/0 (socket says 172.21.15.66:52218) 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62cdd9b700 1 -- 172.21.15.66:0/2947923220 learned_addr learned my addr 172.21.15.66:0/2947923220 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.835+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0103130 0x7f62d01a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ce59c700 1 -- 172.21.15.66:0/2947923220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0103130 msgr2=0x7f62d01a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0103130 0x7f62d01a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ce59c700 1 -- 172.21.15.66:0/2947923220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0102790 msgr2=0x7f62d0199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0102790 0x7f62d0199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ce59c700 1 -- 172.21.15.66:0/2947923220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c00095f0 con 0x7f62d01015a0 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0103130 0x7f62d01a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d0199700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b580 tx=0x7f62c0003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.836+0000 7f62bf7fe700 1 -- 172.21.15.66:0/2947923220 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c001d070 con 0x7f62d01015a0 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.837+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d01a1780 con 0x7f62d01015a0 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.837+0000 7f62bf7fe700 1 -- 172.21.15.66:0/2947923220 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62c000bda0 con 0x7f62d01015a0 2023-12-11T19:32:59.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.837+0000 7f62bf7fe700 1 -- 172.21.15.66:0/2947923220 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0022bd0 con 0x7f62d01015a0 2023-12-11T19:32:59.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.837+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d01a1c00 con 0x7f62d01015a0 2023-12-11T19:32:59.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.840+0000 7f62bf7fe700 1 -- 172.21.15.66:0/2947923220 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f62c0003e10 con 0x7f62d01015a0 2023-12-11T19:32:59.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.841+0000 7f62e841db80 1 Processor -- start 2023-12-11T19:32:59.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.841+0000 7f62bf7fe700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f62b80a1510 0x7f62b80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.841+0000 7f62bf7fe700 1 -- 172.21.15.66:0/2947923220 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f62c00ce330 con 0x7f62d01015a0 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.841+0000 7f62e841db80 1 -- start start 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f62b80a1510 0x7f62b80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62e841db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363340fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62e841db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363341e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62e841db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363342860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62e841db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558363293b00 con 0x5583633423e0 2023-12-11T19:32:59.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62e841db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558363291420 con 0x5583633414f0 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62e841db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55836328cd10 con 0x558363340b90 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62ced9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363342860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62cdd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363341e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62cdd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363341e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52228/0 (socket says 172.21.15.66:52228) 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62cdd9b700 1 -- 172.21.15.66:0/3079876269 learned_addr learned my addr 172.21.15.66:0/3079876269 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.842+0000 7f62ce59c700 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363340fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62cdd9b700 1 -- 172.21.15.66:0/3079876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 msgr2=0x558363340fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363340fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62cdd9b700 1 -- 172.21.15.66:0/3079876269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 msgr2=0x558363342860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363342860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62cdd9b700 1 -- 172.21.15.66:0/3079876269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583631095e0 con 0x5583633414f0 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62ce59c700 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363340fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f62b80a1510 0x7f62b80a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007fd0 tx=0x7f62c800ad90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363341e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62c400bea0 tx=0x7f62c40106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc734757b6d81a0e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62bcff9700 1 -- 172.21.15.66:0/3079876269 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4010ed0 con 0x5583633414f0 2023-12-11T19:32:59.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62bcff9700 1 -- 172.21.15.66:0/3079876269 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62c4015070 con 0x5583633414f0 2023-12-11T19:32:59.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.843+0000 7f62bcff9700 1 -- 172.21.15.66:0/3079876269 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4019750 con 0x5583633414f0 2023-12-11T19:32:59.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 -- 172.21.15.66:0/3079876269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 msgr2=0x558363341e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363341e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62c400bea0 tx=0x7f62c40106b0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 -- 172.21.15.66:0/3079876269 shutdown_connections 2023-12-11T19:32:59.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363342860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363340fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 --2- 172.21.15.66:0/3079876269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363341e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 -- 172.21.15.66:0/3079876269 >> 172.21.15.66:0/3079876269 conn(0x558363267790 msgr2=0x558363334d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 -- 172.21.15.66:0/3079876269 shutdown_connections 2023-12-11T19:32:59.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.844+0000 7f62e841db80 1 -- 172.21.15.66:0/3079876269 wait complete. 2023-12-11T19:32:59.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.845+0000 7f62e841db80 1 -- start start 2023-12-11T19:32:59.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.845+0000 7f62e841db80 1 Processor -- start 2023-12-11T19:32:59.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 0x55836334a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633414f0 0x55836334aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363351ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558363246510 con 0x5583633423e0 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583632461b0 con 0x558363340b90 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55836310b8d0 con 0x5583633414f0 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62e841db80 10 client.0 _ll_register_callbacks cb 0x558363171a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62ce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 0x55836334a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62ce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 0x55836334a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52240/0 (socket says 172.21.15.66:52240) 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62ce59c700 1 -- 172.21.15.66:0/2725896715 learned_addr learned my addr 172.21.15.66:0/2725896715 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633414f0 0x55836334aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.846+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363351ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ce59c700 1 -- 172.21.15.66:0/2725896715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633414f0 msgr2=0x55836334aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633414f0 0x55836334aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ce59c700 1 -- 172.21.15.66:0/2725896715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 msgr2=0x558363351ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363351ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ce59c700 1 -- 172.21.15.66:0/2725896715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d01a1c00 con 0x558363340b90 2023-12-11T19:32:59.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363351ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 0x55836334a970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009fd0 tx=0x7f62c009ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=103a6bfb96253b7f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2725896715 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c009bc40 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2725896715 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f62c009b540 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55836332baf0 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.847+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2725896715 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c009f690 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 10 client.24947 fetch_fsmap learned FSMap version 27 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f62c400bad0 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2725896715 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55836332baf0 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2725896715 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f62c00cd030 con 0x558363340b90 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 10 client.24947 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 10 client.24947 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 20 client.24947 populate_metadata read hostname 'smithi066' 2023-12-11T19:32:59.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x558363356210 0x558363354ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 4 client.24947 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558363307ef0 con 0x558363356210 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.848+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x558363356210 0x558363354ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.849+0000 7f62b5ffb700 10 client.24947 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.849+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x558363356210 0x558363354ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc6b9267c306fca6 server_cookie=d9c8985c99f711c4 in_seq=0 out_seq=0 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2725896715 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558363307ef0 con 0x558363356210 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62b5ffb700 10 client.24947 handle_command_reply: tid=0 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 client.24947 shutdown 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 msgr2=0x55836334a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 0x55836334a970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009fd0 tx=0x7f62c009ba80 comp rx=0 tx=0).stop 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 shutdown_connections 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x558363356210 0x558363354ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558363340b90 0x55836334a970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62b5ffb700 0 client.24947 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:32:59.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x558363351ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 --2- 172.21.15.66:0/2725896715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633414f0 0x55836334aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 >> 172.21.15.66:0/2725896715 conn(0x558363267790 msgr2=0x5583633339f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 shutdown_connections 2023-12-11T19:32:59.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.850+0000 7f62e841db80 1 -- 172.21.15.66:0/2725896715 wait complete. 2023-12-11T19:32:59.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.851+0000 7f62e841db80 20 client.24947 trim_cache size 0 max 16384 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.890+0000 7f62e841db80 1 Processor -- start 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.890+0000 7f62e841db80 1 -- start start 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62e841db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 0x558363334920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62e841db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363351eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62e841db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633423e0 0x558363334e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62e841db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558363307ef0 con 0x558363340b90 2023-12-11T19:32:59.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62e841db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55836328dfa0 con 0x5583633414f0 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62e841db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558363288e00 con 0x5583633423e0 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 0x558363334920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 0x558363334920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49560/0 (socket says 172.21.15.66:49560) 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 -- 172.21.15.66:0/2831930514 learned_addr learned my addr 172.21.15.66:0/2831930514 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633423e0 0x558363334e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363351eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 -- 172.21.15.66:0/2831930514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633423e0 msgr2=0x558363334e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633423e0 0x558363334e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 -- 172.21.15.66:0/2831930514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 msgr2=0x558363351eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363351eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62ce59c700 1 -- 172.21.15.66:0/2831930514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583630b5d90 con 0x558363340b90 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.891+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363351eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.892+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 0x558363334920 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62c0097fb0 tx=0x7f62c0099af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890e9af49786404e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.892+0000 7f62bd7fa700 1 -- 172.21.15.66:0/2831930514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c009bc40 con 0x558363340b90 2023-12-11T19:32:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.892+0000 7f62bd7fa700 1 -- 172.21.15.66:0/2831930514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62c009b540 con 0x558363340b90 2023-12-11T19:32:59.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.892+0000 7f62bd7fa700 1 -- 172.21.15.66:0/2831930514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a3840 con 0x558363340b90 2023-12-11T19:32:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 -- 172.21.15.66:0/2831930514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 msgr2=0x558363334920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 0x558363334920 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62c0097fb0 tx=0x7f62c0099af0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 -- 172.21.15.66:0/2831930514 shutdown_connections 2023-12-11T19:32:59.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583633423e0 0x558363334e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558363340b90 0x558363334920 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 --2- 172.21.15.66:0/2831930514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x558363351eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 -- 172.21.15.66:0/2831930514 >> 172.21.15.66:0/2831930514 conn(0x558363267790 msgr2=0x558363354ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 -- 172.21.15.66:0/2831930514 shutdown_connections 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 -- 172.21.15.66:0/2831930514 wait complete. 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.893+0000 7f62e841db80 1 -- start start 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 Processor -- start 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363351de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x55836334e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x55836334ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55836334f4f0 con 0x5583633423e0 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583633ac2d0 con 0x5583633414f0 2023-12-11T19:32:59.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583633ac460 con 0x558363340b90 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.894+0000 7f62e841db80 10 client.0 _ll_register_callbacks cb 0x558363171a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x55836334ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x55836334ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49568/0 (socket says 172.21.15.66:49568) 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 -- 172.21.15.66:0/2572900359 learned_addr learned my addr 172.21.15.66:0/2572900359 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363351de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x55836334e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 -- 172.21.15.66:0/2572900359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 msgr2=0x558363351de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363351de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 -- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 msgr2=0x55836334e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x55836334e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 -- 172.21.15.66:0/2572900359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c400bad0 con 0x5583633423e0 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62cdd9b700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x55836334e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62ced9d700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x55836334ee10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62c8016c30 tx=0x7f62c8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455de61699e353c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:32:59.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.895+0000 7f62b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2572900359 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8003c00 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2572900359 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f62c8003d60 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62c4015070 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2572900359 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c8019880 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62e841db80 10 client.15117 fetch_fsmap learned FSMap version 27 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55836309ec10 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2572900359 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62c4015070 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2572900359 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f62c80044c0 con 0x5583633423e0 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62e841db80 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62e841db80 10 client.15117 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.896+0000 7f62e841db80 20 client.15117 populate_metadata read hostname 'smithi066' 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.897+0000 7f62e841db80 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x5583633b1840 0x5583633adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.897+0000 7f62e841db80 4 client.15117 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:32:59.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.897+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558363252f80 con 0x5583633b1840 2023-12-11T19:32:59.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.897+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x5583633b1840 0x5583633adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.897+0000 7f62ce59c700 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x5583633b1840 0x5583633adca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8cd388fef44272b5 server_cookie=7407f83a1e552e07 in_seq=0 out_seq=0 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.897+0000 7f62b5ffb700 10 client.15117 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62b5ffb700 1 -- 172.21.15.66:0/2572900359 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558363252f80 con 0x5583633b1840 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62b5ffb700 10 client.15117 handle_command_reply: tid=0 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 client.15117 shutdown 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 msgr2=0x55836334ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x55836334ee10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62c8016c30 tx=0x7f62c8004b40 comp rx=0 tx=0).stop 2023-12-11T19:32:59.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 shutdown_connections 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583633423e0 0x55836334ee10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558363340b90 0x558363351de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x5583633b1840 0x5583633adca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 --2- 172.21.15.66:0/2572900359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583633414f0 0x55836334e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62b5ffb700 0 client.15117 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 >> 172.21.15.66:0/2572900359 conn(0x558363267790 msgr2=0x558363334920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 shutdown_connections 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 1 -- 172.21.15.66:0/2572900359 wait complete. 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.898+0000 7f62e841db80 20 client.15117 trim_cache size 0 max 16384 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:32:59.904 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 3, 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24494, 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 62.384105503999997, 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 96.515058702999994 2023-12-11T19:32:59.905 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f62b80a1510 msgr2=0x7f62b80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f62b80a1510 0x7f62b80a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007fd0 tx=0x7f62c800ad90 comp rx=0 tx=0).stop 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 msgr2=0x7f62d0199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d0199700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b580 tx=0x7f62c0003a50 comp rx=0 tx=0).stop 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 shutdown_connections 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f62b80a1510 0x7f62b80a39d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62d0103130 0x7f62d01a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62d01015a0 0x7f62d0199700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 --2- 172.21.15.66:0/2947923220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62d0102790 0x7f62d0199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:32:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 >> 172.21.15.66:0/2947923220 conn(0x7f62d00fcb10 msgr2=0x7f62d0108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:32:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 shutdown_connections 2023-12-11T19:32:59.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:32:59.899+0000 7f62d49ff700 1 -- 172.21.15.66:0/2947923220 wait complete. 2023-12-11T19:32:59.925 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:32:59.925 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/557291720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:32:59.925 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2538259120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:32:59.925 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:32:59.925 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:32:59.925 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:32:59.926 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:32:59.926 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:32:59.926 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:32:59 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:32:59.927 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-12-11T19:33:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/557291720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2538259120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:32:59 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/557291720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2538259120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:32:59 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 -- 172.21.15.66:0/3102554216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52480ffe70 msgr2=0x7f5248100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 --2- 172.21.15.66:0/3102554216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52480ffe70 0x7f5248100290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5244009990 tx=0x7f5244009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 -- 172.21.15.66:0/3102554216 shutdown_connections 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 --2- 172.21.15.66:0/3102554216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5248101190 0x7f5248101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 --2- 172.21.15.66:0/3102554216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52480ffe70 0x7f5248100290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 --2- 172.21.15.66:0/3102554216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52481007d0 0x7f5248100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 -- 172.21.15.66:0/3102554216 >> 172.21.15.66:0/3102554216 conn(0x7f52480fb9f0 msgr2=0x7f52480fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 -- 172.21.15.66:0/3102554216 shutdown_connections 2023-12-11T19:33:00.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.151+0000 7f524fdff700 1 -- 172.21.15.66:0/3102554216 wait complete. 2023-12-11T19:33:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.153+0000 7f524fdff700 1 Processor -- start 2023-12-11T19:33:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.153+0000 7f524fdff700 1 -- start start 2023-12-11T19:33:00.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.153+0000 7f524fdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52480ffe70 0x7f5248199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.153+0000 7f524fdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 0x7f5248199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524fdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5248101190 0x7f52481a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524fdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52481a11b0 con 0x7f52481007d0 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524fdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52481a12f0 con 0x7f5248101190 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524fdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52481a1480 con 0x7f52480ffe70 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 0x7f5248199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 0x7f5248199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49592/0 (socket says 172.21.15.66:49592) 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 -- 172.21.15.66:0/2548186274 learned_addr learned my addr 172.21.15.66:0/2548186274 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524db9b700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52480ffe70 0x7f5248199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524e39c700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5248101190 0x7f52481a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 -- 172.21.15.66:0/2548186274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52480ffe70 msgr2=0x7f5248199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52480ffe70 0x7f5248199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 -- 172.21.15.66:0/2548186274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5248101190 msgr2=0x7f52481a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5248101190 0x7f52481a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 -- 172.21.15.66:0/2548186274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240009c40 con 0x7f52481007d0 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.154+0000 7f524d39a700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 0x7f5248199bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f523800d950 tx=0x7f523800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.155+0000 7f523effd700 1 -- 172.21.15.66:0/2548186274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238009930 con 0x7f52481007d0 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.155+0000 7f523effd700 1 -- 172.21.15.66:0/2548186274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5238010460 con 0x7f52481007d0 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.155+0000 7f523effd700 1 -- 172.21.15.66:0/2548186274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523800f620 con 0x7f52481007d0 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.155+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52440095f0 con 0x7f52481007d0 2023-12-11T19:33:00.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.155+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52481a1a90 con 0x7f52481007d0 2023-12-11T19:33:00.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.158+0000 7f523effd700 1 -- 172.21.15.66:0/2548186274 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f52380105d0 con 0x7f52481007d0 2023-12-11T19:33:00.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 Processor -- start 2023-12-11T19:33:00.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 -- start start 2023-12-11T19:33:00.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 0x55d206c865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c87050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206be0b00 con 0x55d206c85ce0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206bde420 con 0x55d206c86bd0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.160+0000 7f526380ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206bd9d10 con 0x55d206c85380 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f523effd700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52340a1510 0x7f52340a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 0x55d206c865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524e39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 0x55d206c865f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49594/0 (socket says 172.21.15.66:49594) 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f523effd700 1 -- 172.21.15.66:0/2548186274 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f52380ccc40 con 0x7f52481007d0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524d39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c87050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 -- 172.21.15.66:0/4055131574 learned_addr learned my addr 172.21.15.66:0/4055131574 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524e39c700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52340a1510 0x7f52340a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 -- 172.21.15.66:0/4055131574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 msgr2=0x55d206c857a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c857a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 -- 172.21.15.66:0/4055131574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 msgr2=0x55d206c87050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c87050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524db9b700 1 -- 172.21.15.66:0/4055131574 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d206a565e0 con 0x55d206c85ce0 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.161+0000 7f524d39a700 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c87050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.162+0000 7f524db9b700 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 0x55d206c865f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5244000c00 tx=0x7f5244005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0e92a1a74267f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.162+0000 7f524e39c700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52340a1510 0x7f52340a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5240009550 tx=0x7f524000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.162+0000 7f5233fff700 1 -- 172.21.15.66:0/4055131574 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524401d070 con 0x55d206c85ce0 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.162+0000 7f5233fff700 1 -- 172.21.15.66:0/4055131574 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524401e040 con 0x55d206c85ce0 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.162+0000 7f5233fff700 1 -- 172.21.15.66:0/4055131574 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400b6b0 con 0x55d206c85ce0 2023-12-11T19:33:00.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 -- 172.21.15.66:0/4055131574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 msgr2=0x55d206c865f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 0x55d206c865f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5244000c00 tx=0x7f5244005bc0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 -- 172.21.15.66:0/4055131574 shutdown_connections 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c87050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c857a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 --2- 172.21.15.66:0/4055131574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85ce0 0x55d206c865f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 -- 172.21.15.66:0/4055131574 >> 172.21.15.66:0/4055131574 conn(0x55d206bb4790 msgr2=0x55d206c81d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 -- 172.21.15.66:0/4055131574 shutdown_connections 2023-12-11T19:33:00.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 -- 172.21.15.66:0/4055131574 wait complete. 2023-12-11T19:33:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 -- start start 2023-12-11T19:33:00.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.163+0000 7f526380ab80 1 Processor -- start 2023-12-11T19:33:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c8f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c8f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c86bd0 0x55d206c96650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206b93510 con 0x55d206c85380 2023-12-11T19:33:00.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206b931b0 con 0x55d206c85ce0 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206a588d0 con 0x55d206c86bd0 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f526380ab80 10 client.0 _ll_register_callbacks cb 0x55d206abea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f524db9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c8f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f524d39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c8f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f524e39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c86bd0 0x55d206c96650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f524d39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c8f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52288/0 (socket says 172.21.15.66:52288) 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.164+0000 7f524d39a700 1 -- 172.21.15.66:0/348755319 learned_addr learned my addr 172.21.15.66:0/348755319 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f524e39c700 1 -- 172.21.15.66:0/348755319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 msgr2=0x55d206c8f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f524e39c700 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c8f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f524e39c700 1 -- 172.21.15.66:0/348755319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 msgr2=0x55d206c8f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f524e39c700 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c8f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f524e39c700 1 -- 172.21.15.66:0/348755319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52481a1a90 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f524e39c700 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c86bd0 0x55d206c96650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f524001afd0 tx=0x7f5240004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22feedd6918d66f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f52317fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f52317fa700 1 -- 172.21.15.66:0/348755319 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240029070 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d206c78af0 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.165+0000 7f52317fa700 1 -- 172.21.15.66:0/348755319 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5240009390 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f52317fa700 1 -- 172.21.15.66:0/348755319 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f524000e070 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 10 client.24955 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f524810c4f0 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f52317fa700 1 -- 172.21.15.66:0/348755319 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d206c78af0 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f52317fa700 1 -- 172.21.15.66:0/348755319 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f524002c030 con 0x55d206c86bd0 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 10 client.24955 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 10 client.24955 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 20 client.24955 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206c9ae20 0x55d206c99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 4 client.24955 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.166+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d206c54ef0 con 0x55d206c9ae20 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.167+0000 7f524db9b700 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206c9ae20 0x55d206c99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.167+0000 7f524db9b700 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206c9ae20 0x55d206c99840 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ea16ae8f45e836c server_cookie=920fdb1dd5b8120a in_seq=0 out_seq=0 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.167+0000 7f52317fa700 10 client.24955 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.168+0000 7f52317fa700 1 -- 172.21.15.66:0/348755319 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d206c54ef0 con 0x55d206c9ae20 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.168+0000 7f52317fa700 10 client.24955 handle_command_reply: tid=0 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.168+0000 7f526380ab80 1 client.24955 shutdown 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.168+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c86bd0 msgr2=0x55d206c96650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.168+0000 7f526380ab80 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c86bd0 0x55d206c96650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f524001afd0 tx=0x7f5240004ad0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 shutdown_connections 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206c9ae20 0x55d206c99840 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c86bd0 0x55d206c96650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c8f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 --2- 172.21.15.66:0/348755319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c8f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f52317fa700 0 client.24955 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 >> 172.21.15.66:0/348755319 conn(0x55d206bb4790 msgr2=0x55d206c809f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 shutdown_connections 2023-12-11T19:33:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 1 -- 172.21.15.66:0/348755319 wait complete. 2023-12-11T19:33:00.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.169+0000 7f526380ab80 20 client.24955 trim_cache size 0 max 16384 2023-12-11T19:33:00.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 Processor -- start 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 -- start start 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c8ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c96610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c86bd0 0x55d206c8f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206b75360 con 0x55d206c86bd0 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206b7c830 con 0x55d206c85ce0 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f526380ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206bdafa0 con 0x55d206c85380 2023-12-11T19:33:00.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f524e39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c86bd0 0x55d206c8f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f524db9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c8ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f524e39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c86bd0 0x55d206c8f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49604/0 (socket says 172.21.15.66:49604) 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f524e39c700 1 -- 172.21.15.66:0/2963052183 learned_addr learned my addr 172.21.15.66:0/2963052183 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.188+0000 7f524d39a700 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c96610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f524db9b700 1 -- 172.21.15.66:0/2963052183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 msgr2=0x55d206c96610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f524db9b700 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c96610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f524db9b700 1 -- 172.21.15.66:0/2963052183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c86bd0 msgr2=0x55d206c8f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f524db9b700 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c86bd0 0x55d206c8f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f524db9b700 1 -- 172.21.15.66:0/2963052183 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d206a02d90 con 0x55d206c85380 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f524db9b700 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c8ee80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5244009ce0 tx=0x7f524400b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9950c27ea51cdb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f523cbff700 1 -- 172.21.15.66:0/2963052183 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524401d040 con 0x55d206c85380 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f523cbff700 1 -- 172.21.15.66:0/2963052183 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5244003b10 con 0x55d206c85380 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.189+0000 7f523cbff700 1 -- 172.21.15.66:0/2963052183 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440052b0 con 0x55d206c85380 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 -- 172.21.15.66:0/2963052183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 msgr2=0x55d206c8ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c8ee80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5244009ce0 tx=0x7f524400b5b0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 -- 172.21.15.66:0/2963052183 shutdown_connections 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c86bd0 0x55d206c8f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85380 0x55d206c8ee80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 --2- 172.21.15.66:0/2963052183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c85ce0 0x55d206c96610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 -- 172.21.15.66:0/2963052183 >> 172.21.15.66:0/2963052183 conn(0x55d206bb4790 msgr2=0x55d206c89b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 -- 172.21.15.66:0/2963052183 shutdown_connections 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 -- 172.21.15.66:0/2963052183 wait complete. 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 -- start start 2023-12-11T19:33:00.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.190+0000 7f526380ab80 1 Processor -- start 2023-12-11T19:33:00.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c96290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85ce0 0x55d206c92d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206c937e0 con 0x55d206c85380 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206c93950 con 0x55d206c86bd0 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d206cf0ee0 con 0x55d206c85ce0 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f526380ab80 10 client.0 _ll_register_callbacks cb 0x55d206abea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c96290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c96290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49614/0 (socket says 172.21.15.66:49614) 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 -- 172.21.15.66:0/4091387808 learned_addr learned my addr 172.21.15.66:0/4091387808 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524d39a700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85ce0 0x55d206c92d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524e39c700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 -- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85ce0 msgr2=0x55d206c92d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85ce0 0x55d206c92d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 -- 172.21.15.66:0/4091387808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 msgr2=0x55d206c932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524db9b700 1 -- 172.21.15.66:0/4091387808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d206a565e0 con 0x55d206c85380 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.191+0000 7f524d39a700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85ce0 0x55d206c92d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f524db9b700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c96290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5244005c00 tx=0x7f5244034530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b671eaf3082b0b93 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f52317fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f52317fa700 1 -- 172.21.15.66:0/4091387808 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524401d070 con 0x55d206c85380 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f52317fa700 1 -- 172.21.15.66:0/4091387808 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f52440348d0 con 0x55d206c85380 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f524401e040 con 0x55d206c85380 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f52317fa700 1 -- 172.21.15.66:0/4091387808 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5244034a90 con 0x55d206c85380 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f526380ab80 10 client.15135 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:00.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.192+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d2069ebc10 con 0x55d206c85380 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f52317fa700 1 -- 172.21.15.66:0/4091387808 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f524401e040 con 0x55d206c85380 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f52317fa700 1 -- 172.21.15.66:0/4091387808 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f524400b7d0 con 0x55d206c85380 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f526380ab80 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f526380ab80 10 client.15135 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f526380ab80 20 client.15135 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f526380ab80 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206cf5d20 0x55d206cf2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f526380ab80 4 client.15135 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d206b9ff80 con 0x55d206cf5d20 2023-12-11T19:33:00.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.193+0000 7f524d39a700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206cf5d20 0x55d206cf2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f52317fa700 10 client.15135 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f524d39a700 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206cf5d20 0x55d206cf2180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67da41f8b4daee1a server_cookie=d9fb3da1132ca39f in_seq=0 out_seq=0 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f52317fa700 1 -- 172.21.15.66:0/4091387808 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55d206b9ff80 con 0x55d206cf5d20 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f52317fa700 10 client.15135 handle_command_reply: tid=0 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 client.15135 shutdown 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 msgr2=0x55d206c96290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c96290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5244005c00 tx=0x7f5244034530 comp rx=0 tx=0).stop 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 shutdown_connections 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55d206cf5d20 0x55d206cf2180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d206c86bd0 0x55d206c932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d206c85380 0x55d206c96290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f52317fa700 0 client.15135 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 --2- 172.21.15.66:0/4091387808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d206c85ce0 0x55d206c92d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 >> 172.21.15.66:0/4091387808 conn(0x55d206bb4790 msgr2=0x55d206c8ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 shutdown_connections 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.194+0000 7f526380ab80 1 -- 172.21.15.66:0/4091387808 wait complete. 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f526380ab80 20 client.15135 trim_cache size 0 max 16384 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:00.205 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24514, 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 63.711521501, 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 93.559868714000004 2023-12-11T19:33:00.206 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:00.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52340a1510 msgr2=0x7f52340a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52340a1510 0x7f52340a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5240009550 tx=0x7f524000b040 comp rx=0 tx=0).stop 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 msgr2=0x7f5248199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 0x7f5248199bc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f523800d950 tx=0x7f523800dc60 comp rx=0 tx=0).stop 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 shutdown_connections 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52340a1510 0x7f52340a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5248101190 0x7f52481a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52480ffe70 0x7f5248199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 --2- 172.21.15.66:0/2548186274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52481007d0 0x7f5248199bc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 >> 172.21.15.66:0/2548186274 conn(0x7f52480fb9f0 msgr2=0x7f52480fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 shutdown_connections 2023-12-11T19:33:00.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.195+0000 7f524fdff700 1 -- 172.21.15.66:0/2548186274 wait complete. 2023-12-11T19:33:00.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-11T19:33:00.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 -- 172.21.15.66:0/50526901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80441015a0 msgr2=0x7f80441019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 --2- 172.21.15.66:0/50526901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80441015a0 0x7f80441019c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f80340099c0 tx=0x7f8034009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 -- 172.21.15.66:0/50526901 shutdown_connections 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 --2- 172.21.15.66:0/50526901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8044103130 0x7f80441099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 --2- 172.21.15.66:0/50526901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80441015a0 0x7f80441019c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 --2- 172.21.15.66:0/50526901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8044102790 0x7f8044102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 -- 172.21.15.66:0/50526901 >> 172.21.15.66:0/50526901 conn(0x7f80440fcb10 msgr2=0x7f80440fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 -- 172.21.15.66:0/50526901 shutdown_connections 2023-12-11T19:33:00.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.442+0000 7f80499ff700 1 -- 172.21.15.66:0/50526901 wait complete. 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 Processor -- start 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 -- start start 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80441015a0 0x7f8044199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8044102790 0x7f8044199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 0x7f80441a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441a1270 con 0x7f8044103130 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441a13b0 con 0x7f8044102790 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.443+0000 7f80499ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441a1540 con 0x7f80441015a0 2023-12-11T19:33:00.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 0x7f80441a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 0x7f80441a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49628/0 (socket says 172.21.15.66:49628) 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 -- 172.21.15.66:0/3765759315 learned_addr learned my addr 172.21.15.66:0/3765759315 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80427fc700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8044102790 0x7f8044199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f8042ffd700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80441015a0 0x7f8044199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 -- 172.21.15.66:0/3765759315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80441015a0 msgr2=0x7f8044199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80441015a0 0x7f8044199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 -- 172.21.15.66:0/3765759315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8044102790 msgr2=0x7f8044199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8044102790 0x7f8044199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 -- 172.21.15.66:0/3765759315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340095f0 con 0x7f8044103130 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f8042ffd700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80441015a0 0x7f8044199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.444+0000 7f80437fe700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 0x7f80441a0ca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f803c007ee0 tx=0x7f803c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.445+0000 7f80489fd700 1 -- 172.21.15.66:0/3765759315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00ddd0 con 0x7f8044103130 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.445+0000 7f80489fd700 1 -- 172.21.15.66:0/3765759315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f803c00f040 con 0x7f8044103130 2023-12-11T19:33:00.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.445+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441a1820 con 0x7f8044103130 2023-12-11T19:33:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.445+0000 7f80489fd700 1 -- 172.21.15.66:0/3765759315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0047d0 con 0x7f8044103130 2023-12-11T19:33:00.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.445+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441a1ce0 con 0x7f8044103130 2023-12-11T19:33:00.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.448+0000 7f80489fd700 1 -- 172.21.15.66:0/3765759315 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f803c003a00 con 0x7f8044103130 2023-12-11T19:33:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.449+0000 7f805d25eb80 1 Processor -- start 2023-12-11T19:33:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 -- start start 2023-12-11T19:33:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71380 0x55be7da717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71ce0 0x55be7da725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d9ccb00 con 0x55be7da71380 2023-12-11T19:33:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d9ca420 con 0x55be7da72bd0 2023-12-11T19:33:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f805d25eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d9c5d10 con 0x55be7da71ce0 2023-12-11T19:33:00.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f80489fd700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f802c0a1510 0x7f802c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f80427fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71380 0x55be7da717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f80489fd700 1 -- 172.21.15.66:0/3765759315 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f803c0cc730 con 0x7f8044103130 2023-12-11T19:33:00.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f80427fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71380 0x55be7da717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49638/0 (socket says 172.21.15.66:49638) 2023-12-11T19:33:00.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f80427fc700 1 -- 172.21.15.66:0/902170259 learned_addr learned my addr 172.21.15.66:0/902170259 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.450+0000 7f8042ffd700 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71ce0 0x55be7da725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f80427fc700 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f80437fe700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f802c0a1510 0x7f802c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f8042ffd700 1 -- 172.21.15.66:0/902170259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 msgr2=0x55be7da73050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f8042ffd700 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da73050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f8042ffd700 1 -- 172.21.15.66:0/902170259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71380 msgr2=0x55be7da717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f8042ffd700 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71380 0x55be7da717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f8042ffd700 1 -- 172.21.15.66:0/902170259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be7d8425e0 con 0x55be7da71ce0 2023-12-11T19:33:00.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f80427fc700 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f80437fe700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f802c0a1510 0x7f802c0a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f803c096360 tx=0x7f803c004b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.451+0000 7f8042ffd700 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71ce0 0x55be7da725f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f803400b580 tx=0x7f8034009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f487c15a2955bc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.452+0000 7f8029ffb700 1 -- 172.21.15.66:0/902170259 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400b990 con 0x55be7da71ce0 2023-12-11T19:33:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.452+0000 7f8029ffb700 1 -- 172.21.15.66:0/902170259 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f803401e040 con 0x55be7da71ce0 2023-12-11T19:33:00.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.452+0000 7f8029ffb700 1 -- 172.21.15.66:0/902170259 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034004760 con 0x55be7da71ce0 2023-12-11T19:33:00.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 -- 172.21.15.66:0/902170259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71ce0 msgr2=0x55be7da725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71ce0 0x55be7da725f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f803400b580 tx=0x7f8034009f10 comp rx=0 tx=0).stop 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 -- 172.21.15.66:0/902170259 shutdown_connections 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71380 0x55be7da717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 --2- 172.21.15.66:0/902170259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71ce0 0x55be7da725f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 -- 172.21.15.66:0/902170259 >> 172.21.15.66:0/902170259 conn(0x55be7d9a0790 msgr2=0x55be7da6dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 -- 172.21.15.66:0/902170259 shutdown_connections 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.453+0000 7f805d25eb80 1 -- 172.21.15.66:0/902170259 wait complete. 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.454+0000 7f805d25eb80 1 -- start start 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.454+0000 7f805d25eb80 1 Processor -- start 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 0x55be7da7b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 0x55be7da826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d97f510 con 0x55be7da72bd0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d97f1b0 con 0x55be7da71ce0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d8448d0 con 0x55be7da71380 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f805d25eb80 10 client.0 _ll_register_callbacks cb 0x55be7d8aaa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f80427fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 0x55be7da826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f8042ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 0x55be7da7b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f8042ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 0x55be7da7b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52336/0 (socket says 172.21.15.66:52336) 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f8042ffd700 1 -- 172.21.15.66:0/914878890 learned_addr learned my addr 172.21.15.66:0/914878890 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f80427fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 0x55be7da826d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49652/0 (socket says 172.21.15.66:49652) 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.455+0000 7f80427fc700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8042ffd700 1 -- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 msgr2=0x55be7da7b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8042ffd700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da7b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8042ffd700 1 -- 172.21.15.66:0/914878890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 msgr2=0x55be7da826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8042ffd700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 0x55be7da826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8042ffd700 1 -- 172.21.15.66:0/914878890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804410a330 con 0x55be7da71ce0 2023-12-11T19:33:00.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f80427fc700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 0x55be7da826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:00.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8042ffd700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 0x55be7da7b690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8034000c00 tx=0x7f8034005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9710a2312e65dd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:33:00.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8022ffd700 1 -- 172.21.15.66:0/914878890 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034004500 con 0x55be7da71ce0 2023-12-11T19:33:00.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be7da64af0 con 0x55be7da71ce0 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.456+0000 7f8022ffd700 1 -- 172.21.15.66:0/914878890 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8034004660 con 0x55be7da71ce0 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.457+0000 7f8022ffd700 1 -- 172.21.15.66:0/914878890 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8034004820 con 0x55be7da71ce0 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.457+0000 7f805d25eb80 10 client.24980 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.457+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55be7d7eed90 con 0x55be7da71ce0 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.457+0000 7f8022ffd700 1 -- 172.21.15.66:0/914878890 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f803403b020 con 0x55be7da71ce0 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f8022ffd700 1 -- 172.21.15.66:0/914878890 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8034003df0 con 0x55be7da71ce0 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f805d25eb80 10 client.24980 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f805d25eb80 10 client.24980 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f805d25eb80 20 client.24980 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f805d25eb80 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7da86900 0x55be7da856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f805d25eb80 4 client.24980 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be7da40ef0 con 0x55be7da86900 2023-12-11T19:33:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.458+0000 7f80427fc700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7da86900 0x55be7da856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.459+0000 7f8022ffd700 10 client.24980 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.459+0000 7f80427fc700 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7da86900 0x55be7da856a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1254d27db43b653b server_cookie=c76b0133668f106e in_seq=0 out_seq=0 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f8022ffd700 1 -- 172.21.15.66:0/914878890 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be7da40ef0 con 0x55be7da86900 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f8022ffd700 10 client.24980 handle_command_reply: tid=0 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 client.24980 shutdown 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 msgr2=0x55be7da7b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 0x55be7da7b690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8034000c00 tx=0x7f8034005270 comp rx=0 tx=0).stop 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 shutdown_connections 2023-12-11T19:33:00.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da72bd0 0x55be7da826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da7b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7da86900 0x55be7da856a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 --2- 172.21.15.66:0/914878890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71ce0 0x55be7da7b690 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f8022ffd700 0 client.24980 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 >> 172.21.15.66:0/914878890 conn(0x55be7d9a0790 msgr2=0x55be7da6c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 shutdown_connections 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.460+0000 7f805d25eb80 1 -- 172.21.15.66:0/914878890 wait complete. 2023-12-11T19:33:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.461+0000 7f805d25eb80 20 client.24980 trim_cache size 0 max 16384 2023-12-11T19:33:00.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.479+0000 7f805d25eb80 1 Processor -- start 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.479+0000 7f805d25eb80 1 -- start start 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71380 0x55be7da72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da72bd0 0x55be7da75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f805d25eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d9c1e00 con 0x55be7da71ce0 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f805d25eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d96ab50 con 0x55be7da71380 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f805d25eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7d961360 con 0x55be7da72bd0 2023-12-11T19:33:00.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da825f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49660/0 (socket says 172.21.15.66:49660) 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 -- 172.21.15.66:0/1297424671 learned_addr learned my addr 172.21.15.66:0/1297424671 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f80427fc700 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da72bd0 0x55be7da75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 -- 172.21.15.66:0/1297424671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da72bd0 msgr2=0x55be7da75b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da72bd0 0x55be7da75b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 -- 172.21.15.66:0/1297424671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71380 msgr2=0x55be7da72fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71380 0x55be7da72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 -- 172.21.15.66:0/1297424671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be7d7d7c10 con 0x55be7da71ce0 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f80427fc700 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da72bd0 0x55be7da75b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:00.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.480+0000 7f8042ffd700 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da825f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80340041d0 tx=0x7f8034009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8c85a3e702e59a server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.481+0000 7f802a7fc700 1 -- 172.21.15.66:0/1297424671 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400bba0 con 0x55be7da71ce0 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.481+0000 7f802a7fc700 1 -- 172.21.15.66:0/1297424671 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f803403b040 con 0x55be7da71ce0 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.481+0000 7f802a7fc700 1 -- 172.21.15.66:0/1297424671 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400f900 con 0x55be7da71ce0 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.481+0000 7f805d25eb80 1 -- 172.21.15.66:0/1297424671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 msgr2=0x55be7da825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.481+0000 7f805d25eb80 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da825f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80340041d0 tx=0x7f8034009310 comp rx=0 tx=0).stop 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 -- 172.21.15.66:0/1297424671 shutdown_connections 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da71380 0x55be7da72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da72bd0 0x55be7da75b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 --2- 172.21.15.66:0/1297424671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da825f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 -- 172.21.15.66:0/1297424671 >> 172.21.15.66:0/1297424671 conn(0x55be7d9a0790 msgr2=0x55be7da856a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 -- 172.21.15.66:0/1297424671 shutdown_connections 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 -- 172.21.15.66:0/1297424671 wait complete. 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.482+0000 7f805d25eb80 1 -- start start 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 Processor -- start 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da82510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da7efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7da7fa80 con 0x55be7da71ce0 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7da7fbf0 con 0x55be7da72bd0 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be7dadca10 con 0x55be7da71380 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f805d25eb80 10 client.0 _ll_register_callbacks cb 0x55be7d8aaa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f8042ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da7efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.483+0000 7f80427fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da82510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f80427fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da82510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:58596/0 (socket says 172.21.15.66:58596) 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da7efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49670/0 (socket says 172.21.15.66:49670) 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f80427fc700 1 -- 172.21.15.66:0/93478548 learned_addr learned my addr 172.21.15.66:0/93478548 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f80427fc700 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 -- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 msgr2=0x55be7da82510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da82510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 -- 172.21.15.66:0/93478548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 msgr2=0x55be7da7f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da7f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 -- 172.21.15.66:0/93478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be7d7eed90 con 0x55be7da71ce0 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8042ffd700 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da7efb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8034004360 tx=0x7f8034005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae6c572fdc2af54 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:00.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.484+0000 7f8022ffd700 1 -- 172.21.15.66:0/93478548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034003a20 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f803401e040 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f8022ffd700 1 -- 172.21.15.66:0/93478548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8034000e40 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f8022ffd700 1 -- 172.21.15.66:0/93478548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8034003b80 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f805d25eb80 10 client.15153 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55be7d815200 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f8022ffd700 1 -- 172.21.15.66:0/93478548 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f803404f020 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f8022ffd700 1 -- 172.21.15.66:0/93478548 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8034004590 con 0x55be7da71ce0 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.485+0000 7f805d25eb80 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.486+0000 7f805d25eb80 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.486+0000 7f805d25eb80 20 client.15153 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.486+0000 7f805d25eb80 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7dae1750 0x55be7daddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.486+0000 7f805d25eb80 4 client.15153 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:00.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.486+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be7d98bf80 con 0x55be7dae1750 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.486+0000 7f80427fc700 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7dae1750 0x55be7daddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f8022ffd700 10 client.15153 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f80427fc700 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7dae1750 0x55be7daddc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b337fcf774be719 server_cookie=9497e810afac2be1 in_seq=0 out_seq=0 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f8022ffd700 1 -- 172.21.15.66:0/93478548 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55be7d98bf80 con 0x55be7dae1750 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f8022ffd700 10 client.15153 handle_command_reply: tid=0 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 client.15153 shutdown 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 msgr2=0x55be7da7efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da7efb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8034004360 tx=0x7f8034005cc0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 shutdown_connections 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55be7dae1750 0x55be7daddc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55be7da72bd0 0x55be7da7f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be7da71380 0x55be7da82510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f8022ffd700 0 client.15153 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 --2- 172.21.15.66:0/93478548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55be7da71ce0 0x55be7da7efb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 >> 172.21.15.66:0/93478548 conn(0x55be7d9a0790 msgr2=0x55be7da856a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 shutdown_connections 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.487+0000 7f805d25eb80 1 -- 172.21.15.66:0/93478548 wait complete. 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f805d25eb80 20 client.15153 trim_cache size 0 max 16384 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24547, 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:00.493 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:00.494 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 64.988998879999997, 2023-12-11T19:33:00.494 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:00.494 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:00.494 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:00.494 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 84.347855867000007 2023-12-11T19:33:00.494 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f802c0a1510 msgr2=0x7f802c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f802c0a1510 0x7f802c0a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f803c096360 tx=0x7f803c004b40 comp rx=0 tx=0).stop 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 msgr2=0x7f80441a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 0x7f80441a0ca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f803c007ee0 tx=0x7f803c00d3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 shutdown_connections 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f802c0a1510 0x7f802c0a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8044103130 0x7f80441a0ca0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80441015a0 0x7f8044199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 --2- 172.21.15.66:0/3765759315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8044102790 0x7f8044199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 >> 172.21.15.66:0/3765759315 conn(0x7f80440fcb10 msgr2=0x7f8044107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 shutdown_connections 2023-12-11T19:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.488+0000 7f80499ff700 1 -- 172.21.15.66:0/3765759315 wait complete. 2023-12-11T19:33:00.515 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-11T19:33:00.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 -- 172.21.15.66:0/1703923795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc075610 msgr2=0x7f28dc073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1703923795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc075610 0x7f28dc073a60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009990 tx=0x7f28cc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 -- 172.21.15.66:0/1703923795 shutdown_connections 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1703923795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28dc10db70 0x7f28dc10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1703923795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc075610 0x7f28dc073a60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1703923795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28dc074030 0x7f28dc0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 -- 172.21.15.66:0/1703923795 >> 172.21.15.66:0/1703923795 conn(0x7f28dc0fcb10 msgr2=0x7f28dc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 -- 172.21.15.66:0/1703923795 shutdown_connections 2023-12-11T19:33:00.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.737+0000 7f28e13ff700 1 -- 172.21.15.66:0/1703923795 wait complete. 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 Processor -- start 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 -- start start 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28dc074030 0x7f28dc19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28dc075610 0x7f28dc19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 0x7f28dc1a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc1a55a0 con 0x7f28dc10db70 2023-12-11T19:33:00.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc1a56e0 con 0x7f28dc074030 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.738+0000 7f28e13ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc1a5870 con 0x7f28dc075610 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 0x7f28dc1a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 0x7f28dc1a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49694/0 (socket says 172.21.15.66:49694) 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 -- 172.21.15.66:0/1252255306 learned_addr learned my addr 172.21.15.66:0/1252255306 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28da7fc700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28dc075610 0x7f28dc19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28daffd700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28dc074030 0x7f28dc19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 -- 172.21.15.66:0/1252255306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28dc075610 msgr2=0x7f28dc19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28dc075610 0x7f28dc19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 -- 172.21.15.66:0/1252255306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28dc074030 msgr2=0x7f28dc19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28dc074030 0x7f28dc19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 -- 172.21.15.66:0/1252255306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0095f0 con 0x7f28dc10db70 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.739+0000 7f28db7fe700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 0x7f28dc1a4fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f28d400ef30 tx=0x7f28d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.740+0000 7f28c3fff700 1 -- 172.21.15.66:0/1252255306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4018070 con 0x7f28dc10db70 2023-12-11T19:33:00.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.740+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc1a5af0 con 0x7f28dc10db70 2023-12-11T19:33:00.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.740+0000 7f28c3fff700 1 -- 172.21.15.66:0/1252255306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28d400f040 con 0x7f28dc10db70 2023-12-11T19:33:00.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.740+0000 7f28c3fff700 1 -- 172.21.15.66:0/1252255306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d400cbc0 con 0x7f28dc10db70 2023-12-11T19:33:00.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.740+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc1a6000 con 0x7f28dc10db70 2023-12-11T19:33:00.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.743+0000 7f28c3fff700 1 -- 172.21.15.66:0/1252255306 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f28d4007500 con 0x7f28dc10db70 2023-12-11T19:33:00.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.744+0000 7f28c3fff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f28c40a1510 0x7f28c40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.744+0000 7f28f4c60b80 1 Processor -- start 2023-12-11T19:33:00.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28c3fff700 1 -- 172.21.15.66:0/1252255306 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28d40cc900 con 0x7f28dc10db70 2023-12-11T19:33:00.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 -- start start 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28daffd700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f28c40a1510 0x7f28c40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f4590b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f4590c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4590d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f45866b00 con 0x561f4590cbd0 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f45864420 con 0x561f4590b380 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4585fd10 con 0x561f4590bce0 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4590d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4590d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49708/0 (socket says 172.21.15.66:49708) 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28db7fe700 1 -- 172.21.15.66:0/4276263298 learned_addr learned my addr 172.21.15.66:0/4276263298 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.745+0000 7f28da7fc700 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f4590c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28daffd700 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f4590b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28db7fe700 1 -- 172.21.15.66:0/4276263298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 msgr2=0x561f4590c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28db7fe700 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f4590c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28db7fe700 1 -- 172.21.15.66:0/4276263298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 msgr2=0x561f4590b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28db7fe700 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f4590b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28db7fe700 1 -- 172.21.15.66:0/4276263298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f456dc5e0 con 0x561f4590cbd0 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28daffd700 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f4590b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28daffd700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f28c40a1510 0x7f28c40a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005910 tx=0x7f28cc00b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28db7fe700 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4590d050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28d4096580 tx=0x7f28d4099750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6108c1078a1728 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28c17fa700 1 -- 172.21.15.66:0/4276263298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d409a210 con 0x561f4590cbd0 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28c17fa700 1 -- 172.21.15.66:0/4276263298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28d4084040 con 0x561f4590cbd0 2023-12-11T19:33:00.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.746+0000 7f28c17fa700 1 -- 172.21.15.66:0/4276263298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d400cda0 con 0x561f4590cbd0 2023-12-11T19:33:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 -- 172.21.15.66:0/4276263298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 msgr2=0x561f4590d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4590d050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28d4096580 tx=0x7f28d4099750 comp rx=0 tx=0).stop 2023-12-11T19:33:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 -- 172.21.15.66:0/4276263298 shutdown_connections 2023-12-11T19:33:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f4590b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4590d050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/4276263298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f4590c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 -- 172.21.15.66:0/4276263298 >> 172.21.15.66:0/4276263298 conn(0x561f4583a790 msgr2=0x561f45907d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 -- 172.21.15.66:0/4276263298 shutdown_connections 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.747+0000 7f28f4c60b80 1 -- 172.21.15.66:0/4276263298 wait complete. 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.748+0000 7f28f4c60b80 1 -- start start 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.748+0000 7f28f4c60b80 1 Processor -- start 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 0x561f45915180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f459156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590cbd0 0x561f4591c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f45819510 con 0x561f4590b380 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f458191b0 con 0x561f4590cbd0 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f456de8d0 con 0x561f4590bce0 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28f4c60b80 10 client.0 _ll_register_callbacks cb 0x561f45744a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28daffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 0x561f45915180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28daffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 0x561f45915180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49716/0 (socket says 172.21.15.66:49716) 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28daffd700 1 -- 172.21.15.66:0/3789077798 learned_addr learned my addr 172.21.15.66:0/3789077798 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28da7fc700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f459156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.749+0000 7f28db7fe700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590cbd0 0x561f4591c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28daffd700 1 -- 172.21.15.66:0/3789077798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 msgr2=0x561f459156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28daffd700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f459156c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28daffd700 1 -- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590cbd0 msgr2=0x561f4591c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28daffd700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590cbd0 0x561f4591c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28daffd700 1 -- 172.21.15.66:0/3789077798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc1a6000 con 0x561f4590b380 2023-12-11T19:33:00.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28da7fc700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f459156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28daffd700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 0x561f45915180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28cc015000 tx=0x7f28cc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c1278a4913b6cb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28beff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28beff5700 1 -- 172.21.15.66:0/3789077798 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00b910 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28beff5700 1 -- 172.21.15.66:0/3789077798 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28cc00ba70 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.750+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f458feaf0 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28beff5700 1 -- 172.21.15.66:0/3789077798 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28cc00bc30 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 10 client.15165 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f45688d90 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28beff5700 1 -- 172.21.15.66:0/3789077798 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f458feaf0 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28beff5700 1 -- 172.21.15.66:0/3789077798 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f28cc025470 con 0x561f4590b380 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 10 client.15165 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 20 client.15165 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:00.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f45920de0 0x561f4591f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 4 client.15165 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:00.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.751+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f458daef0 con 0x561f45920de0 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.752+0000 7f28da7fc700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f45920de0 0x561f4591f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.752+0000 7f28da7fc700 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f45920de0 0x561f4591f860 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d5be62be36048a server_cookie=1d03b8de4aadab3b in_seq=0 out_seq=0 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.752+0000 7f28beff5700 10 client.15165 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.753+0000 7f28beff5700 1 -- 172.21.15.66:0/3789077798 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f458daef0 con 0x561f45920de0 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.753+0000 7f28beff5700 10 client.15165 handle_command_reply: tid=0 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.753+0000 7f28f4c60b80 1 client.15165 shutdown 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.753+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 msgr2=0x561f45915180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.753+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 0x561f45915180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28cc015000 tx=0x7f28cc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 shutdown_connections 2023-12-11T19:33:00.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f45920de0 0x561f4591f860 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590cbd0 0x561f4591c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590b380 0x561f45915180 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/3789077798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f459156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 >> 172.21.15.66:0/3789077798 conn(0x561f4583a790 msgr2=0x561f459069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.754+0000 7f28beff5700 0 client.15165 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.755+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 shutdown_connections 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.755+0000 7f28f4c60b80 1 -- 172.21.15.66:0/3789077798 wait complete. 2023-12-11T19:33:00.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.755+0000 7f28f4c60b80 20 client.15165 trim_cache size 0 max 16384 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 Processor -- start 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 -- start start 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590b380 0x561f45908030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590bce0 0x561f4591c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f45804b50 con 0x561f4590cbd0 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4590af20 con 0x561f4590bce0 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.774+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f45913780 con 0x561f4590b380 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49722/0 (socket says 172.21.15.66:49722) 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 -- 172.21.15.66:0/1817211607 learned_addr learned my addr 172.21.15.66:0/1817211607 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28da7fc700 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590bce0 0x561f4591c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28daffd700 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590b380 0x561f45908030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 -- 172.21.15.66:0/1817211607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590b380 msgr2=0x561f45908030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590b380 0x561f45908030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 -- 172.21.15.66:0/1817211607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590bce0 msgr2=0x561f4591c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590bce0 0x561f4591c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 -- 172.21.15.66:0/1817211607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f45671c10 con 0x561f4590cbd0 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28da7fc700 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590bce0 0x561f4591c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.775+0000 7f28db7fe700 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591f860 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f28d4099a50 tx=0x7f28d4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979bc17787087a19 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28c1ffb700 1 -- 172.21.15.66:0/1817211607 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d409dc60 con 0x561f4590cbd0 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28c1ffb700 1 -- 172.21.15.66:0/1817211607 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28d4099b10 con 0x561f4590cbd0 2023-12-11T19:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28c1ffb700 1 -- 172.21.15.66:0/1817211607 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40ca4c0 con 0x561f4590cbd0 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 -- 172.21.15.66:0/1817211607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 msgr2=0x561f4591f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591f860 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f28d4099a50 tx=0x7f28d4009ed0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 -- 172.21.15.66:0/1817211607 shutdown_connections 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591f860 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590b380 0x561f45908030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/1817211607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590bce0 0x561f4591c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.776+0000 7f28f4c60b80 1 -- 172.21.15.66:0/1817211607 >> 172.21.15.66:0/1817211607 conn(0x561f4583a790 msgr2=0x561f45907920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.777+0000 7f28f4c60b80 1 -- 172.21.15.66:0/1817211607 shutdown_connections 2023-12-11T19:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.777+0000 7f28f4c60b80 1 -- 172.21.15.66:0/1817211607 wait complete. 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.777+0000 7f28f4c60b80 1 -- start start 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.777+0000 7f28f4c60b80 1 Processor -- start 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f459168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f45916e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4591bc40 con 0x561f4590cbd0 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4591bdb0 con 0x561f4590b380 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4591bf20 con 0x561f4590bce0 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28f4c60b80 10 client.0 _ll_register_callbacks cb 0x561f45744a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49724/0 (socket says 172.21.15.66:49724) 2023-12-11T19:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 -- 172.21.15.66:0/2727641747 learned_addr learned my addr 172.21.15.66:0/2727641747 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28da7fc700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f45916e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 -- 172.21.15.66:0/2727641747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 msgr2=0x561f45916e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f45916e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28daffd700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f459168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 -- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 msgr2=0x561f459168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f459168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 -- 172.21.15.66:0/2727641747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d0011440 con 0x561f4590cbd0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28daffd700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f459168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28db7fe700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591b700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28d400c060 tx=0x7f28d400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28616d4b8745ce2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.778+0000 7f28beff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28beff5700 1 -- 172.21.15.66:0/2727641747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d409dc60 con 0x561f4590cbd0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28beff5700 1 -- 172.21.15.66:0/2727641747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28d409ae40 con 0x561f4590cbd0 2023-12-11T19:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28d4084040 con 0x561f4590cbd0 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28beff5700 1 -- 172.21.15.66:0/2727641747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d40a1e10 con 0x561f4590cbd0 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 10 client.15171 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f456dc5e0 con 0x561f4590cbd0 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28beff5700 1 -- 172.21.15.66:0/2727641747 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28d4084040 con 0x561f4590cbd0 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28beff5700 1 -- 172.21.15.66:0/2727641747 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f28d4099d70 con 0x561f4590cbd0 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 20 client.15171 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f4597afb0 0x561f45977440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 4 client.15171 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.779+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561f45825f80 con 0x561f4597afb0 2023-12-11T19:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.780+0000 7f28daffd700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f4597afb0 0x561f45977440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.780+0000 7f28daffd700 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f4597afb0 0x561f45977440 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31944f7139867fd1 server_cookie=f5b2f62799b52c14 in_seq=0 out_seq=0 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.780+0000 7f28beff5700 10 client.15171 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.780+0000 7f28beff5700 1 -- 172.21.15.66:0/2727641747 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561f45825f80 con 0x561f4597afb0 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.780+0000 7f28beff5700 10 client.15171 handle_command_reply: tid=0 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.780+0000 7f28f4c60b80 1 client.15171 shutdown 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 msgr2=0x561f4591b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591b700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28d400c060 tx=0x7f28d400cfd0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 shutdown_connections 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f4597afb0 0x561f45977440 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f4590b380 0x561f459168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f4590cbd0 0x561f4591b700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 --2- 172.21.15.66:0/2727641747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f4590bce0 0x561f45916e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 >> 172.21.15.66:0/2727641747 conn(0x561f4583a790 msgr2=0x561f4591f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28beff5700 0 client.15171 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 shutdown_connections 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 1 -- 172.21.15.66:0/2727641747 wait complete. 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.781+0000 7f28f4c60b80 20 client.15171 trim_cache size 0 max 16384 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:00.787 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24575, 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 68.302473864000007, 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 74.910105158999997 2023-12-11T19:33:00.788 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f28c40a1510 msgr2=0x7f28c40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f28c40a1510 0x7f28c40a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005910 tx=0x7f28cc00b610 comp rx=0 tx=0).stop 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 msgr2=0x7f28dc1a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 0x7f28dc1a4fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f28d400ef30 tx=0x7f28d400c3a0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 shutdown_connections 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f28c40a1510 0x7f28c40a39d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28dc074030 0x7f28dc19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28dc10db70 0x7f28dc1a4fd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 --2- 172.21.15.66:0/1252255306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28dc075610 0x7f28dc19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 >> 172.21.15.66:0/1252255306 conn(0x7f28dc0fcb10 msgr2=0x7f28dc100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 shutdown_connections 2023-12-11T19:33:00.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:00.782+0000 7f28e13ff700 1 -- 172.21.15.66:0/1252255306 wait complete. 2023-12-11T19:33:00.799 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:33:00.799 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:33:00.799 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:00 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:00.802 INFO:tasks.cephadm:Setup complete, yielding 2023-12-11T19:33:00.802 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-11T19:33:00.813 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:33:00.814 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph orch status' 2023-12-11T19:33:01.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:33:01.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:33:01.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:00 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:01.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:33:01.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:33:01.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:00 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:33:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:02.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:01 smithi167 ceph-mon[102750]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:02.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:01 smithi066 ceph-mon[97920]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:01 smithi070 ceph-mon[104044]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:03.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 -- 172.21.15.66:0/1234493991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 msgr2=0x7fbadc102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:03.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 --2- 172.21.15.66:0/1234493991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc102c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbac80099c0 tx=0x7fbac8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 -- 172.21.15.66:0/1234493991 shutdown_connections 2023-12-11T19:33:03.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 --2- 172.21.15.66:0/1234493991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc1031c0 0x7fbadc109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 --2- 172.21.15.66:0/1234493991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbadc101620 0x7fbadc101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 --2- 172.21.15.66:0/1234493991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc102c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 -- 172.21.15.66:0/1234493991 >> 172.21.15.66:0/1234493991 conn(0x7fbadc0fcbe0 msgr2=0x7fbadc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:03.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 -- 172.21.15.66:0/1234493991 shutdown_connections 2023-12-11T19:33:03.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.160+0000 7fbae2c67700 1 -- 172.21.15.66:0/1234493991 wait complete. 2023-12-11T19:33:03.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.161+0000 7fbae2c67700 1 Processor -- start 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 -- start start 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc101620 0x7fbadc1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbadc1031c0 0x7fbadc19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc19f000 con 0x7fbadc102820 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc19f140 con 0x7fbadc101620 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae2c67700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc19f2d0 con 0x7fbadc1031c0 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbadbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbadbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56472/0 (socket says 172.21.15.66:56472) 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbadbfff700 1 -- 172.21.15.66:0/3130220768 learned_addr learned my addr 172.21.15.66:0/3130220768 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.162+0000 7fbae1204700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbadc1031c0 0x7fbadc19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbae0a03700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc101620 0x7fbadc1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbadbfff700 1 -- 172.21.15.66:0/3130220768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbadc1031c0 msgr2=0x7fbadc19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbadbfff700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbadc1031c0 0x7fbadc19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbadbfff700 1 -- 172.21.15.66:0/3130220768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc101620 msgr2=0x7fbadc1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbadbfff700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc101620 0x7fbadc1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbadbfff700 1 -- 172.21.15.66:0/3130220768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac80095f0 con 0x7fbadc102820 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbadbfff700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc197a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b580 tx=0x7fbac8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac801d070 con 0x7fbadc102820 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbac800bd30 con 0x7fbadc102820 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc19f550 con 0x7fbadc102820 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8022e50 con 0x7fbadc102820 2023-12-11T19:33:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.163+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc19f9c0 con 0x7fbadc102820 2023-12-11T19:33:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.165+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc108bf0 con 0x7fbadc102820 2023-12-11T19:33:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.166+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbac8004040 con 0x7fbadc102820 2023-12-11T19:33:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.168+0000 7fbad9ffb700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbacc0a1590 0x7fbacc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.168+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbac80ce2a0 con 0x7fbadc102820 2023-12-11T19:33:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.168+0000 7fbae0a03700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbacc0a1590 0x7fbacc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:03.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.169+0000 7fbae0a03700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbacc0a1590 0x7fbacc0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbad0005fd0 tx=0x7fbad0005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:03.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.172+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbac805d2f0 con 0x7fbadc102820 2023-12-11T19:33:03.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.402+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbadc061a10 con 0x7fbacc0a1590 2023-12-11T19:33:03.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.404+0000 7fbad9ffb700 1 -- 172.21.15.66:0/3130220768 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbadc061a10 con 0x7fbacc0a1590 2023-12-11T19:33:03.405 INFO:teuthology.orchestra.run.smithi066.stdout:Backend: cephadm 2023-12-11T19:33:03.405 INFO:teuthology.orchestra.run.smithi066.stdout:Available: Yes 2023-12-11T19:33:03.405 INFO:teuthology.orchestra.run.smithi066.stdout:Paused: No 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbacc0a1590 msgr2=0x7fbacc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbacc0a1590 0x7fbacc0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbad0005fd0 tx=0x7fbad0005dd0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 msgr2=0x7fbadc197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc197a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b580 tx=0x7fbac8003b00 comp rx=0 tx=0).stop 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 shutdown_connections 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbacc0a1590 0x7fbacc0a3a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc101620 0x7fbadc1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbadc1031c0 0x7fbadc19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 --2- 172.21.15.66:0/3130220768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbadc102820 0x7fbadc197a00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.406+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 >> 172.21.15.66:0/3130220768 conn(0x7fbadc0fcbe0 msgr2=0x7fbadc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:03.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.407+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 shutdown_connections 2023-12-11T19:33:03.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:03.407+0000 7fbae2c67700 1 -- 172.21.15.66:0/3130220768 wait complete. 2023-12-11T19:33:03.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-11T19:33:03.974 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:03 smithi066 ceph-mon[97920]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:04.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:03 smithi167 ceph-mon[102750]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:04.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:03 smithi070 ceph-mon[104044]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:04.700 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:04.998 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:04 smithi066 ceph-mon[97920]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:05.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:04 smithi167 ceph-mon[102750]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:05.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:04 smithi070 ceph-mon[104044]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:05.977 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:05 smithi066 ceph-mon[97920]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:06.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:05 smithi167 ceph-mon[102750]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:06.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:05 smithi070 ceph-mon[104044]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:06.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 -- 172.21.15.66:0/964274319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534101620 msgr2=0x7f4534101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:06.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 --2- 172.21.15.66:0/964274319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534101620 0x7f4534101a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45300099c0 tx=0x7f4530009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 -- 172.21.15.66:0/964274319 shutdown_connections 2023-12-11T19:33:06.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 --2- 172.21.15.66:0/964274319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45341031c0 0x7f4534109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 --2- 172.21.15.66:0/964274319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534101620 0x7f4534101a40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 --2- 172.21.15.66:0/964274319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4534102820 0x7f4534102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 -- 172.21.15.66:0/964274319 >> 172.21.15.66:0/964274319 conn(0x7f45340fcbe0 msgr2=0x7f45340ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 -- 172.21.15.66:0/964274319 shutdown_connections 2023-12-11T19:33:06.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.244+0000 7f453b887700 1 -- 172.21.15.66:0/964274319 wait complete. 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.245+0000 7f453b887700 1 Processor -- start 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 -- start start 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4534101620 0x7f4534197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 0x7f4534197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45341031c0 0x7f453419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453419f050 con 0x7f4534102820 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453419f190 con 0x7f45341031c0 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f453b887700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453419f320 con 0x7f4534101620 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f4538e22700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 0x7f4534197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f4538e22700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 0x7f4534197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56500/0 (socket says 172.21.15.66:56500) 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f4538e22700 1 -- 172.21.15.66:0/3681570032 learned_addr learned my addr 172.21.15.66:0/3681570032 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:06.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f4539623700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4534101620 0x7f4534197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.246+0000 7f4539e24700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45341031c0 0x7f453419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f4538e22700 1 -- 172.21.15.66:0/3681570032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4534101620 msgr2=0x7f4534197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f4538e22700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4534101620 0x7f4534197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f4538e22700 1 -- 172.21.15.66:0/3681570032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45341031c0 msgr2=0x7f453419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:06.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f4538e22700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45341031c0 0x7f453419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f4538e22700 1 -- 172.21.15.66:0/3681570032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300095f0 con 0x7f4534102820 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f4538e22700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 0x7f4534197aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f452400d980 tx=0x7f452400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.247+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45240098d0 con 0x7f4534102820 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.248+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4524010460 con 0x7f4534102820 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.248+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453419f600 con 0x7f4534102820 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.248+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400f620 con 0x7f4534102820 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.248+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453419fab0 con 0x7f4534102820 2023-12-11T19:33:06.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.249+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4534108bf0 con 0x7f4534102820 2023-12-11T19:33:06.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.251+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4524009a30 con 0x7f4534102820 2023-12-11T19:33:06.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.252+0000 7f452a7fc700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45200a1510 0x7f45200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:06.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.252+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f45240ccc60 con 0x7f4534102820 2023-12-11T19:33:06.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.252+0000 7f4539623700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45200a1510 0x7f45200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:06.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.253+0000 7f4539623700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45200a1510 0x7f45200a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4530006010 tx=0x7f4530003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:06.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.256+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4524096120 con 0x7f4534102820 2023-12-11T19:33:06.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.476+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f453402d380 con 0x7f45200a1510 2023-12-11T19:33:06.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.490+0000 7f452a7fc700 1 -- 172.21.15.66:0/3681570032 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f453402d380 con 0x7f45200a1510 2023-12-11T19:33:06.492 INFO:teuthology.orchestra.run.smithi066.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-11T19:33:06.492 INFO:teuthology.orchestra.run.smithi066.stdout:mds.a smithi066 running (84s) 6s ago 83s 17.4M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 c8a30422417b 2023-12-11T19:33:06.492 INFO:teuthology.orchestra.run.smithi066.stdout:mds.b smithi070 running (80s) 7s ago 80s 29.6M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 e0de8f6167d3 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.c smithi167 running (97s) 7s ago 96s 17.7M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 21f9f2999920 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.d smithi066 running (106s) 6s ago 106s 13.9M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 21701851f8cd 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.e smithi070 running (103s) 7s ago 103s 18.1M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 fb1c463d68fb 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.f smithi167 running (100s) 7s ago 99s 18.8M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 aa05e653c4a3 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.g smithi066 running (87s) 6s ago 86s 17.5M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 62b8fae8fd6b 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.h smithi070 running (93s) 7s ago 93s 18.9M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 2f1c6a06191f 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.i smithi167 running (90s) 7s ago 90s 18.6M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 6f60c3f01b7a 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.j smithi066 running (109s) 6s ago 109s 14.0M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 7c65d7fccb3d 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.k smithi070 running (113s) 7s ago 112s 18.2M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 c6679829221b 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mds.l smithi167 running (116s) 7s ago 115s 17.5M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 393d0681aebd 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.x smithi066 *:9283,8765 running (9m) 6s ago 9m 531M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 227ea30622be 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.y smithi070 *:8443,8765 running (7m) 7s ago 7m 453M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 bf015c9dc388 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.z smithi167 *:8443,8765 running (7m) 7s ago 7m 453M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 8f55d6791a3d 2023-12-11T19:33:06.493 INFO:teuthology.orchestra.run.smithi066.stdout:mon.a smithi066 running (9m) 6s ago 9m 98.3M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 6739963f110c 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:mon.b smithi070 running (7m) 7s ago 7m 66.6M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 84d5cd727a8d 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:mon.c smithi167 running (8m) 7s ago 8m 65.3M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 20afe80bde55 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 smithi066 running (7m) 6s ago 7m 69.9M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 c57b9582ba86 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 smithi070 running (6m) 7s ago 6m 77.6M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 55699ba478af 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 smithi167 running (6m) 7s ago 6m 73.7M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 0b6ed12fe269 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 smithi066 running (5m) 6s ago 5m 63.3M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 a262092b4e7d 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 smithi070 running (5m) 7s ago 5m 62.7M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 57067c496251 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 smithi167 running (5m) 7s ago 5m 65.7M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 1a73917a8d76 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 smithi066 running (4m) 6s ago 4m 60.1M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 2095ab034a5c 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 smithi070 running (4m) 7s ago 4m 64.7M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 2fc23b2b175a 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.8 smithi167 running (3m) 7s ago 3m 61.5M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 738ea808fbef 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.9 smithi066 running (3m) 6s ago 3m 58.6M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 f25ed349a943 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.10 smithi070 running (2m) 7s ago 2m 63.2M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 10de2af0344f 2023-12-11T19:33:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:osd.11 smithi167 running (2m) 7s ago 2m 53.4M 4317M 19.0.0-119-ga62fda73 0b0a5e4a17a7 ad9af274b833 2023-12-11T19:33:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.492+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45200a1510 msgr2=0x7f45200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.492+0000 7f453b887700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45200a1510 0x7f45200a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4530006010 tx=0x7f4530003740 comp rx=0 tx=0).stop 2023-12-11T19:33:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 msgr2=0x7f4534197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 0x7f4534197aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f452400d980 tx=0x7f452400dc90 comp rx=0 tx=0).stop 2023-12-11T19:33:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 shutdown_connections 2023-12-11T19:33:06.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45200a1510 0x7f45200a39d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45341031c0 0x7f453419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4534101620 0x7f4534197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 --2- 172.21.15.66:0/3681570032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4534102820 0x7f4534197aa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 >> 172.21.15.66:0/3681570032 conn(0x7f45340fcbe0 msgr2=0x7f4534105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 shutdown_connections 2023-12-11T19:33:06.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:06.493+0000 7f453b887700 1 -- 172.21.15.66:0/3681570032 wait complete. 2023-12-11T19:33:07.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-11T19:33:07.800 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:07 smithi167 ceph-mon[102750]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:33:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:07 smithi167 ceph-mon[102750]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:08.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:07 smithi066 ceph-mon[97920]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:33:08.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:07 smithi066 ceph-mon[97920]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:08.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:07 smithi070 ceph-mon[104044]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-11T19:33:08.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:07 smithi070 ceph-mon[104044]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:09.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 -- 172.21.15.66:0/1956842778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45281031c0 msgr2=0x7f4528109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 --2- 172.21.15.66:0/1956842778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45281031c0 0x7f4528109a70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f452000b0d0 tx=0x7f452000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 -- 172.21.15.66:0/1956842778 shutdown_connections 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 --2- 172.21.15.66:0/1956842778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45281031c0 0x7f4528109a70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 --2- 172.21.15.66:0/1956842778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4528101620 0x7f4528101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 --2- 172.21.15.66:0/1956842778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528102820 0x7f4528102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 -- 172.21.15.66:0/1956842778 >> 172.21.15.66:0/1956842778 conn(0x7f45280fcbe0 msgr2=0x7f45280ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 -- 172.21.15.66:0/1956842778 shutdown_connections 2023-12-11T19:33:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.344+0000 7f452dd67700 1 -- 172.21.15.66:0/1956842778 wait complete. 2023-12-11T19:33:09.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.345+0000 7f452dd67700 1 Processor -- start 2023-12-11T19:33:09.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.345+0000 7f452dd67700 1 -- start start 2023-12-11T19:33:09.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.345+0000 7f452dd67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528101620 0x7f4528199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:09.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.345+0000 7f452dd67700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 0x7f4528199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f452dd67700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45281031c0 0x7f45281a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f452dd67700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281a1250 con 0x7f4528102820 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f452dd67700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281a1390 con 0x7f4528101620 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f452dd67700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281a1520 con 0x7f45281031c0 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 0x7f4528199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 0x7f4528199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56522/0 (socket says 172.21.15.66:56522) 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 -- 172.21.15.66:0/3101085223 learned_addr learned my addr 172.21.15.66:0/3101085223 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f45277fe700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528101620 0x7f4528199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4527fff700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45281031c0 0x7f45281a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 -- 172.21.15.66:0/3101085223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45281031c0 msgr2=0x7f45281a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45281031c0 0x7f45281a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 -- 172.21.15.66:0/3101085223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528101620 msgr2=0x7f4528199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528101620 0x7f4528199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 -- 172.21.15.66:0/3101085223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520009d00 con 0x7f4528102820 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f45277fe700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528101620 0x7f4528199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4527fff700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45281031c0 0x7f45281a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.346+0000 7f4526ffd700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 0x7f4528199c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f451c00d980 tx=0x7f451c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.347+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c009b40 con 0x7f4528102820 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.347+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f451c00b6e0 con 0x7f4528102820 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.347+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c00f620 con 0x7f4528102820 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.347+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45281a1800 con 0x7f4528102820 2023-12-11T19:33:09.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.347+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45281a1d00 con 0x7f4528102820 2023-12-11T19:33:09.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.348+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452804f290 con 0x7f4528102820 2023-12-11T19:33:09.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.350+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f451c010460 con 0x7f4528102820 2023-12-11T19:33:09.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.351+0000 7f4524ff9700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45100a1590 0x7f45100a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:09.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.352+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f451c0ccd10 con 0x7f4528102820 2023-12-11T19:33:09.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.352+0000 7f45277fe700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45100a1590 0x7f45100a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:09.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.352+0000 7f45277fe700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45100a1590 0x7f45100a3a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4528102680 tx=0x7f451800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:09.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.356+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f451c0d2020 con 0x7f4528102820 2023-12-11T19:33:09.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.575+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4528074f20 con 0x7f45100a1590 2023-12-11T19:33:09.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.581+0000 7f4524ff9700 1 -- 172.21.15.66:0/3101085223 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f4528074f20 con 0x7f45100a1590 2023-12-11T19:33:09.583 INFO:teuthology.orchestra.run.smithi066.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-11T19:33:09.583 INFO:teuthology.orchestra.run.smithi066.stdout:mds.all 12/12 10s ago 2m smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi070=b;smithi070=e;smithi070=h;smithi070=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-12-11T19:33:09.583 INFO:teuthology.orchestra.run.smithi066.stdout:mgr 3/3 10s ago 7m smithi066=x;smithi070=y;smithi167=z;count:3 2023-12-11T19:33:09.583 INFO:teuthology.orchestra.run.smithi066.stdout:mon 3/3 10s ago 8m smithi066:172.21.15.66=a;smithi070:172.21.15.70=b;smithi167:172.21.15.167=c;count:3 2023-12-11T19:33:09.583 INFO:teuthology.orchestra.run.smithi066.stdout:osd 12 10s ago - 2023-12-11T19:33:09.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45100a1590 msgr2=0x7f45100a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:09.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45100a1590 0x7f45100a3a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4528102680 tx=0x7f451800a680 comp rx=0 tx=0).stop 2023-12-11T19:33:09.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 msgr2=0x7f4528199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:09.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 0x7f4528199c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f451c00d980 tx=0x7f451c00dd40 comp rx=0 tx=0).stop 2023-12-11T19:33:09.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 shutdown_connections 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f45100a1590 0x7f45100a3a50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4528101620 0x7f4528199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f45281031c0 0x7f45281a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 --2- 172.21.15.66:0/3101085223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4528102820 0x7f4528199c50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 >> 172.21.15.66:0/3101085223 conn(0x7f45280fcbe0 msgr2=0x7f4528073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 shutdown_connections 2023-12-11T19:33:09.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:09.584+0000 7f452dd67700 1 -- 172.21.15.66:0/3101085223 wait complete. 2023-12-11T19:33:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:09 smithi167 ceph-mon[102750]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:10.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:09 smithi066 ceph-mon[97920]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:10.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-11T19:33:10.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:09 smithi070 ceph-mon[104044]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:10.933 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:11.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:10 smithi167 ceph-mon[102750]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:11.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:10 smithi066 ceph-mon[97920]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:11.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:10 smithi070 ceph-mon[104044]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:11.774 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:11 smithi066 ceph-mon[97920]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:11.775 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:11 smithi066 ceph-mon[97920]: mgrmap e18: x(active, since 9m), standbys: y, z 2023-12-11T19:33:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:11 smithi167 ceph-mon[102750]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:11 smithi167 ceph-mon[102750]: mgrmap e18: x(active, since 9m), standbys: y, z 2023-12-11T19:33:12.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:11 smithi070 ceph-mon[104044]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:12.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:11 smithi070 ceph-mon[104044]: mgrmap e18: x(active, since 9m), standbys: y, z 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/3397079287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c102820 msgr2=0x7fba9c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/3397079287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c102820 0x7fba9c102c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0099c0 tx=0x7fba8c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/3397079287 shutdown_connections 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/3397079287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba9c1031c0 0x7fba9c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/3397079287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba9c101620 0x7fba9c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/3397079287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c102820 0x7fba9c102c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/3397079287 >> 172.21.15.66:0/3397079287 conn(0x7fba9c0fcbe0 msgr2=0x7fba9c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/3397079287 shutdown_connections 2023-12-11T19:33:12.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.468+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/3397079287 wait complete. 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.469+0000 7fbaa3f2c700 1 Processor -- start 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 -- start start 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 0x7fba9c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba9c102820 0x7fba9c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba9c1031c0 0x7fba9c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c1a1150 con 0x7fba9c101620 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c1a1290 con 0x7fba9c1031c0 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa3f2c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c1a1420 con 0x7fba9c102820 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa1cc8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 0x7fba9c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa1cc8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 0x7fba9c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45306/0 (socket says 172.21.15.66:45306) 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa1cc8700 1 -- 172.21.15.66:0/1951832404 learned_addr learned my addr 172.21.15.66:0/1951832404 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa14c7700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba9c102820 0x7fba9c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.470+0000 7fbaa24c9700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba9c1031c0 0x7fba9c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:12.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa1cc8700 1 -- 172.21.15.66:0/1951832404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba9c102820 msgr2=0x7fba9c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:12.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa1cc8700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba9c102820 0x7fba9c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa1cc8700 1 -- 172.21.15.66:0/1951832404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba9c1031c0 msgr2=0x7fba9c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:12.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa1cc8700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba9c1031c0 0x7fba9c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa1cc8700 1 -- 172.21.15.66:0/1951832404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0095f0 con 0x7fba9c101620 2023-12-11T19:33:12.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa1cc8700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 0x7fba9c199610 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c9c0 tx=0x7fba9800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98007ac0 con 0x7fba9c101620 2023-12-11T19:33:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba98007c20 con 0x7fba9c101620 2023-12-11T19:33:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98018660 con 0x7fba9c101620 2023-12-11T19:33:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c1a1700 con 0x7fba9c101620 2023-12-11T19:33:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.471+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c1a1bb0 con 0x7fba9c101620 2023-12-11T19:33:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.472+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c04f290 con 0x7fba9c101620 2023-12-11T19:33:12.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.474+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fba9800f450 con 0x7fba9c101620 2023-12-11T19:33:12.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.476+0000 7fba92ffd700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fba880a1500 0x7fba880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.476+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fba980ccc40 con 0x7fba9c101620 2023-12-11T19:33:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.476+0000 7fbaa14c7700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fba880a1500 0x7fba880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:12.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.476+0000 7fbaa14c7700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fba880a1500 0x7fba880a39c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009990 tx=0x7fba8c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:12.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.479+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fba98096260 con 0x7fba9c101620 2023-12-11T19:33:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.699+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fba9c074f20 con 0x7fba880a1500 2023-12-11T19:33:12.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.701+0000 7fba92ffd700 1 -- 172.21.15.66:0/1951832404 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fba9c074f20 con 0x7fba880a1500 2023-12-11T19:33:12.702 INFO:teuthology.orchestra.run.smithi066.stdout:HOST ADDR LABELS STATUS 2023-12-11T19:33:12.702 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 172.21.15.66 2023-12-11T19:33:12.702 INFO:teuthology.orchestra.run.smithi066.stdout:smithi070 172.21.15.70 2023-12-11T19:33:12.702 INFO:teuthology.orchestra.run.smithi066.stdout:smithi167 172.21.15.167 2023-12-11T19:33:12.703 INFO:teuthology.orchestra.run.smithi066.stdout:3 hosts in cluster 2023-12-11T19:33:12.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fba880a1500 msgr2=0x7fba880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:12.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fba880a1500 0x7fba880a39c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009990 tx=0x7fba8c000bc0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 msgr2=0x7fba9c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:12.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 0x7fba9c199610 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fba9800c9c0 tx=0x7fba9800ccd0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 shutdown_connections 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fba880a1500 0x7fba880a39c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba9c1031c0 0x7fba9c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba9c101620 0x7fba9c199610 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 --2- 172.21.15.66:0/1951832404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba9c102820 0x7fba9c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 >> 172.21.15.66:0/1951832404 conn(0x7fba9c0fcbe0 msgr2=0x7fba9c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 shutdown_connections 2023-12-11T19:33:12.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:12.703+0000 7fbaa3f2c700 1 -- 172.21.15.66:0/1951832404 wait complete. 2023-12-11T19:33:13.230 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-11T19:33:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:13 smithi167 ceph-mon[102750]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-11T19:33:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:13 smithi167 ceph-mon[102750]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:14.032 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:14.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:13 smithi066 ceph-mon[97920]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-11T19:33:14.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:13 smithi066 ceph-mon[97920]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:14.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:13 smithi070 ceph-mon[104044]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-11T19:33:14.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:13 smithi070 ceph-mon[104044]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 -- 172.21.15.66:0/2830934118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d0102820 msgr2=0x7f37d0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 --2- 172.21.15.66:0/2830934118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d0102820 0x7f37d0102c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f37c00099c0 tx=0x7f37c0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 -- 172.21.15.66:0/2830934118 shutdown_connections 2023-12-11T19:33:15.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 --2- 172.21.15.66:0/2830934118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d0101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 --2- 172.21.15.66:0/2830934118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37d01031c0 0x7f37d0109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 --2- 172.21.15.66:0/2830934118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d0102820 0x7f37d0102c80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 -- 172.21.15.66:0/2830934118 >> 172.21.15.66:0/2830934118 conn(0x7f37d00fcbe0 msgr2=0x7f37d00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 -- 172.21.15.66:0/2830934118 shutdown_connections 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.568+0000 7f37d781c700 1 -- 172.21.15.66:0/2830934118 wait complete. 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.569+0000 7f37d781c700 1 Processor -- start 2023-12-11T19:33:15.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.569+0000 7f37d781c700 1 -- start start 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.569+0000 7f37d781c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.569+0000 7f37d55b8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.569+0000 7f37d55b8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:39822/0 (socket says 172.21.15.66:39822) 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37d0102820 0x7f37d0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d01031c0 0x7f37d01a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d01a11f0 con 0x7f37d01031c0 2023-12-11T19:33:15.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d01a1330 con 0x7f37d0101620 2023-12-11T19:33:15.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 -- 172.21.15.66:0/595242530 learned_addr learned my addr 172.21.15.66:0/595242530 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:15.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d01a14c0 con 0x7f37d0102820 2023-12-11T19:33:15.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d5db9700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d01031c0 0x7f37d01a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d4db7700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37d0102820 0x7f37d0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 -- 172.21.15.66:0/595242530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37d0102820 msgr2=0x7f37d0199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37d0102820 0x7f37d0199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 -- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d01031c0 msgr2=0x7f37d01a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d01031c0 0x7f37d01a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c00095f0 con 0x7f37d0101620 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d5db9700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d01031c0 0x7f37d01a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d55b8700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d01996b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d980 tx=0x7f37cc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:15.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc004020 con 0x7f37d0101620 2023-12-11T19:33:15.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37cc009730 con 0x7f37d0101620 2023-12-11T19:33:15.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc003aa0 con 0x7f37d0101620 2023-12-11T19:33:15.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d01a17a0 con 0x7f37d0101620 2023-12-11T19:33:15.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.571+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d01a1c50 con 0x7f37d0101620 2023-12-11T19:33:15.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.572+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d0193d20 con 0x7f37d0101620 2023-12-11T19:33:15.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.575+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f37cc0098a0 con 0x7f37d0101620 2023-12-11T19:33:15.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.576+0000 7f37c67fc700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f37bc0a1480 0x7f37bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:15.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.576+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f37cc0cc9d0 con 0x7f37d0101620 2023-12-11T19:33:15.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.576+0000 7f37d4db7700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f37bc0a1480 0x7f37bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:15.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.577+0000 7f37d4db7700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f37bc0a1480 0x7f37bc0a3940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c010 tx=0x7f37c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:15.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.579+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f37cc095ff0 con 0x7f37d0101620 2023-12-11T19:33:15.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.801+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f37d002d380 con 0x7f37bc0a1480 2023-12-11T19:33:15.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.802+0000 7f37c67fc700 1 -- 172.21.15.66:0/595242530 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f37d002d380 con 0x7f37bc0a1480 2023-12-11T19:33:15.804 INFO:teuthology.orchestra.run.smithi066.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-11T19:33:15.804 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-12-11T19:33:15.804 INFO:teuthology.orchestra.run.smithi066.stdout:smithi070 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002P480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-12-11T19:33:15.804 INFO:teuthology.orchestra.run.smithi066.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-11T19:33:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f37bc0a1480 msgr2=0x7f37bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f37bc0a1480 0x7f37bc0a3940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c010 tx=0x7f37c0005fb0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 msgr2=0x7f37d01996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d01996b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d980 tx=0x7f37cc00dc90 comp rx=0 tx=0).stop 2023-12-11T19:33:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 shutdown_connections 2023-12-11T19:33:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f37bc0a1480 0x7f37bc0a3940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37d0101620 0x7f37d01996b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37d01031c0 0x7f37d01a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 --2- 172.21.15.66:0/595242530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37d0102820 0x7f37d0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 >> 172.21.15.66:0/595242530 conn(0x7f37d00fcbe0 msgr2=0x7f37d01080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 shutdown_connections 2023-12-11T19:33:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:15.805+0000 7f37d781c700 1 -- 172.21.15.66:0/595242530 wait complete. 2023-12-11T19:33:16.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:15 smithi167 ceph-mon[102750]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:33:16.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:15 smithi066 ceph-mon[97920]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:33:16.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:15 smithi070 ceph-mon[104044]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:33:16.326 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-11T19:33:16.336 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:33:16.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-11T19:33:17.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:16 smithi167 ceph-mon[102750]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:17.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:16 smithi066 ceph-mon[97920]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:17.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:16 smithi070 ceph-mon[104044]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T19:33:17.137 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:18.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:17 smithi167 ceph-mon[102750]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-11T19:33:18.042 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:17 smithi066 ceph-mon[97920]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-11T19:33:18.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:17 smithi070 ceph-mon[104044]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-11T19:33:18.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.700+0000 7fd2f4544700 1 -- 172.21.15.66:0/1693298333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1049d0 msgr2=0x7fd2ec106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:18.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.700+0000 7fd2f4544700 1 --2- 172.21.15.66:0/1693298333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1049d0 0x7fd2ec106dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009990 tx=0x7fd2dc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 -- 172.21.15.66:0/1693298333 shutdown_connections 2023-12-11T19:33:18.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 --2- 172.21.15.66:0/1693298333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 --2- 172.21.15.66:0/1693298333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec107300 0x7fd2ec109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 --2- 172.21.15.66:0/1693298333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1049d0 0x7fd2ec106dc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 -- 172.21.15.66:0/1693298333 >> 172.21.15.66:0/1693298333 conn(0x7fd2ec0fbad0 msgr2=0x7fd2ec0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:18.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 -- 172.21.15.66:0/1693298333 shutdown_connections 2023-12-11T19:33:18.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.701+0000 7fd2f4544700 1 -- 172.21.15.66:0/1693298333 wait complete. 2023-12-11T19:33:18.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 Processor -- start 2023-12-11T19:33:18.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 -- start start 2023-12-11T19:33:18.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:18.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec1049d0 0x7fd2ec100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec107300 0x7fd2ec100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec100d30 con 0x7fd2ec1020a0 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec100ea0 con 0x7fd2ec107300 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.702+0000 7fd2f4544700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1a1500 con 0x7fd2ec1049d0 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec1019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45348/0 (socket says 172.21.15.66:45348) 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 -- 172.21.15.66:0/2506369237 learned_addr learned my addr 172.21.15.66:0/2506369237 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f2ae1700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec107300 0x7fd2ec100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f1adf700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec1049d0 0x7fd2ec100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 -- 172.21.15.66:0/2506369237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec1049d0 msgr2=0x7fd2ec100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec1049d0 0x7fd2ec100050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 -- 172.21.15.66:0/2506369237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec107300 msgr2=0x7fd2ec100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec107300 0x7fd2ec100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 -- 172.21.15.66:0/2506369237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0095f0 con 0x7fd2ec1020a0 2023-12-11T19:33:18.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f2ae1700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec107300 0x7fd2ec100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:18.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.703+0000 7fd2f22e0700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec1019b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800d980 tx=0x7fd2e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:18.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.704+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800bb60 con 0x7fd2ec1020a0 2023-12-11T19:33:18.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.704+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1a1790 con 0x7fd2ec1020a0 2023-12-11T19:33:18.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.704+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2e8007640 con 0x7fd2ec1020a0 2023-12-11T19:33:18.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.704+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8004d60 con 0x7fd2ec1020a0 2023-12-11T19:33:18.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.704+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1a1ce0 con 0x7fd2ec1020a0 2023-12-11T19:33:18.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.705+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec0fd650 con 0x7fd2ec1020a0 2023-12-11T19:33:18.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.707+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd2e8004020 con 0x7fd2ec1020a0 2023-12-11T19:33:18.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.708+0000 7fd2e37fe700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d80a1500 0x7fd2d80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:18.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.709+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2e80ccbe0 con 0x7fd2ec1020a0 2023-12-11T19:33:18.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.709+0000 7fd2f1adf700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d80a1500 0x7fd2d80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:18.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.709+0000 7fd2f1adf700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d80a1500 0x7fd2d80a39c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0098e0 tx=0x7fd2dc005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:18.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.713+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd2e8096200 con 0x7fd2ec1020a0 2023-12-11T19:33:18.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.968+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd2ec04f290 con 0x7fd2ec1020a0 2023-12-11T19:33:18.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.969+0000 7fd2e37fe700 1 -- 172.21.15.66:0/2506369237 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2967 (secure 0 0 0) 0x7fd2e8016020 con 0x7fd2ec1020a0 2023-12-11T19:33:18.970 INFO:teuthology.orchestra.run.smithi066.stdout:e27 2023-12-11T19:33:18.970 INFO:teuthology.orchestra.run.smithi066.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T19:33:18.970 INFO:teuthology.orchestra.run.smithi066.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T19:33:18.970 INFO:teuthology.orchestra.run.smithi066.stdout:legacy client fscid: 1 2023-12-11T19:33:18.970 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:Filesystem 'cephfs' (1) 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:fs_name cephfs 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 27 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:created 2023-12-11T19:31:51.764249+0000 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:modified 2023-12-11T19:31:59.541695+0000 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:tableserver 0 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:root 0 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:session_timeout 300 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:session_autoclose 300 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:max_file_size 1099511627776 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:max_xattr_size 65536 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:required_client_features {} 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:last_failure 0 2023-12-11T19:33:18.971 INFO:teuthology.orchestra.run.smithi066.stdout:last_failure_osd_epoch 0 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:max_mds 5 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:in 0,1,2,3,4 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:up {0=24575,1=24547,2=24514,3=24494,4=24467} 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:failed 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:damaged 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:stopped 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:data_pools [3] 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:metadata_pool 2 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:inline_data disabled 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:balancer 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:bal_rank_mask -1 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:standby_count_wanted 0 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b{0:24575} state up:active seq 3 addr [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.972 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.h{0:24533} state up:standby-replay seq 1 addr [v2:172.21.15.70:6836/280227891,v1:172.21.15.70:6837/280227891] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.i{1:24547} state up:active seq 6 addr [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c{1:24526} state up:standby-replay seq 1 addr [v2:172.21.15.167:6836/2775450491,v1:172.21.15.167:6837/2775450491] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.f{2:24514} state up:active seq 9 addr [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.l{2:24463} state up:standby-replay seq 1 addr [v2:172.21.15.167:6832/1537942709,v1:172.21.15.167:6833/1537942709] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.e{3:24494} state up:active seq 10 addr [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a{3:14685} state up:standby-replay seq 1 addr [v2:172.21.15.66:6840/1874182059,v1:172.21.15.66:6841/1874182059] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.k{4:24467} state up:active seq 13 addr [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.g{4:14673} state up:standby-replay seq 1 addr [v2:172.21.15.66:6838/2148751938,v1:172.21.15.66:6839/2148751938] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:Standby daemons: 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.j{-1:14583} state up:standby seq 1 addr [v2:172.21.15.66:6834/1528570111,v1:172.21.15.66:6835/1528570111] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.973 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d{-1:14595} state up:standby seq 1 addr [v2:172.21.15.66:6836/4055829654,v1:172.21.15.66:6837/4055829654] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T19:33:18.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d80a1500 msgr2=0x7fd2d80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:18.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d80a1500 0x7fd2d80a39c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0098e0 tx=0x7fd2dc005e80 comp rx=0 tx=0).stop 2023-12-11T19:33:18.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 msgr2=0x7fd2ec1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:18.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec1019b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800d980 tx=0x7fd2e800dd40 comp rx=0 tx=0).stop 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 shutdown_connections 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d80a1500 0x7fd2d80a39c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec107300 0x7fd2ec100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1020a0 0x7fd2ec1019b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 --2- 172.21.15.66:0/2506369237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec1049d0 0x7fd2ec100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 >> 172.21.15.66:0/2506369237 conn(0x7fd2ec0fbad0 msgr2=0x7fd2ec103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 shutdown_connections 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:18.971+0000 7fd2f4544700 1 -- 172.21.15.66:0/2506369237 wait complete. 2023-12-11T19:33:18.975 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:33:19.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-11T19:33:20.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:19 smithi167 ceph-mon[102750]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-11T19:33:20.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:19 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2506369237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-11T19:33:20.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:19 smithi066 ceph-mon[97920]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-11T19:33:20.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:19 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2506369237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-11T19:33:20.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:19 smithi070 ceph-mon[104044]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-11T19:33:20.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:19 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2506369237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-11T19:33:20.239 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:22.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:21 smithi167 ceph-mon[102750]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-11T19:33:22.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:21 smithi066 ceph-mon[97920]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 -- 172.21.15.66:0/3218724185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac8102820 msgr2=0x7feac8102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 --2- 172.21.15.66:0/3218724185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac8102820 0x7feac8102c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feabc0099c0 tx=0x7feabc009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 -- 172.21.15.66:0/3218724185 shutdown_connections 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 --2- 172.21.15.66:0/3218724185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8101620 0x7feac8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 --2- 172.21.15.66:0/3218724185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac81031c0 0x7feac8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 --2- 172.21.15.66:0/3218724185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac8102820 0x7feac8102c80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 -- 172.21.15.66:0/3218724185 >> 172.21.15.66:0/3218724185 conn(0x7feac80fcbe0 msgr2=0x7feac80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 -- 172.21.15.66:0/3218724185 shutdown_connections 2023-12-11T19:33:22.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.067+0000 7feacda8d700 1 -- 172.21.15.66:0/3218724185 wait complete. 2023-12-11T19:33:22.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.068+0000 7feacda8d700 1 Processor -- start 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.068+0000 7feacda8d700 1 -- start start 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feacda8d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac8101620 0x7feac8075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feacda8d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8102820 0x7feac80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feacda8d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 0x7feac8073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feacda8d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80746a0 con 0x7feac81031c0 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feacda8d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8074810 con 0x7feac8102820 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feacda8d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac81a14d0 con 0x7feac8101620 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feac7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 0x7feac8073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:22.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feac7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 0x7feac8073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57928/0 (socket says 172.21.15.66:57928) 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feac7fff700 1 -- 172.21.15.66:0/612883726 learned_addr learned my addr 172.21.15.66:0/612883726 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feac6ffd700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8102820 0x7feac80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.069+0000 7feac77fe700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac8101620 0x7feac8075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac7fff700 1 -- 172.21.15.66:0/612883726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac8101620 msgr2=0x7feac8075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac7fff700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac8101620 0x7feac8075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac7fff700 1 -- 172.21.15.66:0/612883726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8102820 msgr2=0x7feac80739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac7fff700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8102820 0x7feac80739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac7fff700 1 -- 172.21.15.66:0/612883726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0095f0 con 0x7feac81031c0 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac77fe700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac8101620 0x7feac8075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac7fff700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 0x7feac8073f00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feac0007ee0 tx=0x7feac000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:22.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0017070 con 0x7feac81031c0 2023-12-11T19:33:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feac000f040 con 0x7feac81031c0 2023-12-11T19:33:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0005710 con 0x7feac81031c0 2023-12-11T19:33:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac81a1760 con 0x7feac81031c0 2023-12-11T19:33:22.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.070+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac81a1c10 con 0x7feac81031c0 2023-12-11T19:33:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.071+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac8108af0 con 0x7feac81031c0 2023-12-11T19:33:22.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.073+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7feac0005870 con 0x7feac81031c0 2023-12-11T19:33:22.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.075+0000 7feac4ff9700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feab00a1500 0x7feab00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:22.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.075+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feac00cc500 con 0x7feac81031c0 2023-12-11T19:33:22.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.075+0000 7feac77fe700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feab00a1500 0x7feab00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:22.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.075+0000 7feac77fe700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feab00a1500 0x7feab00a39c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feac8102680 tx=0x7feab8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:22.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.079+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7feac0095b20 con 0x7feac81031c0 2023-12-11T19:33:22.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:21 smithi070 ceph-mon[104044]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-11T19:33:22.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.294+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7feac81084d0 con 0x7feac81031c0 2023-12-11T19:33:22.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.295+0000 7feac4ff9700 1 -- 172.21.15.66:0/612883726 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5269 (secure 0 0 0) 0x7feac0099140 con 0x7feac81031c0 2023-12-11T19:33:22.296 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 88 2023-12-11T19:33:22.296 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:33:22.296 INFO:teuthology.orchestra.run.smithi066.stdout:created 2023-12-11T19:23:09.144370+0000 2023-12-11T19:33:22.296 INFO:teuthology.orchestra.run.smithi066.stdout:modified 2023-12-11T19:32:11.575017+0000 2023-12-11T19:33:22.296 INFO:teuthology.orchestra.run.smithi066.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:crush_version 26 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:full_ratio 0.95 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:backfillfull_ratio 0.9 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:nearfull_ratio 0.85 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:require_min_compat_client luminous 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:min_compat_client luminous 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:require_osd_release reef 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:stretch_mode_enabled false 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:max_osd 12 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6802/345409379,v1:172.21.15.66:6803/345409379] [v2:172.21.15.66:6804/345409379,v1:172.21.15.66:6805/345409379] exists,up d5da1c6d-b901-47aa-b2e3-4e377f852890 2023-12-11T19:33:22.297 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6800/1682554189,v1:172.21.15.70:6801/1682554189] [v2:172.21.15.70:6802/1682554189,v1:172.21.15.70:6803/1682554189] exists,up 820e245f-dede-4543-a91d-d7ba2568ff7d 2023-12-11T19:33:22.298 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/440232508,v1:172.21.15.167:6801/440232508] [v2:172.21.15.167:6802/440232508,v1:172.21.15.167:6803/440232508] exists,up 0cb5fd9c-e264-4947-85e5-413b230c4575 2023-12-11T19:33:22.298 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6810/1707492505,v1:172.21.15.66:6811/1707492505] [v2:172.21.15.66:6812/1707492505,v1:172.21.15.66:6813/1707492505] exists,up c2c58a66-528c-490e-8f4d-6fdb5be75536 2023-12-11T19:33:22.298 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6808/161576683,v1:172.21.15.70:6809/161576683] [v2:172.21.15.70:6810/161576683,v1:172.21.15.70:6811/161576683] exists,up 181213ac-13a3-4539-9435-df0f7299186b 2023-12-11T19:33:22.298 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/1023707142,v1:172.21.15.167:6809/1023707142] [v2:172.21.15.167:6810/1023707142,v1:172.21.15.167:6811/1023707142] exists,up df3fbfc0-cd26-4ac1-906c-28bd60f02c38 2023-12-11T19:33:22.298 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6818/7519239,v1:172.21.15.66:6819/7519239] [v2:172.21.15.66:6820/7519239,v1:172.21.15.66:6821/7519239] exists,up bb558c6d-a82c-4255-a040-4432a98c2aa1 2023-12-11T19:33:22.298 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6816/4038002078,v1:172.21.15.70:6817/4038002078] [v2:172.21.15.70:6818/4038002078,v1:172.21.15.70:6819/4038002078] exists,up ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/2623463019,v1:172.21.15.167:6817/2623463019] [v2:172.21.15.167:6818/2623463019,v1:172.21.15.167:6819/2623463019] exists,up 5d35b137-1e21-4e15-b0b8-4f0c6c693e73 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6826/969904470,v1:172.21.15.66:6827/969904470] [v2:172.21.15.66:6828/969904470,v1:172.21.15.66:6829/969904470] exists,up 241627a2-87be-4277-9306-48d9f64595a9 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6824/2336173720,v1:172.21.15.70:6825/2336173720] [v2:172.21.15.70:6826/2336173720,v1:172.21.15.70:6827/2336173720] exists,up afbe1b4e-b374-46a0-884f-89c5a4cf84ab 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/819959610,v1:172.21.15.167:6825/819959610] [v2:172.21.15.167:6826/819959610,v1:172.21.15.167:6827/819959610] exists,up af4e5d83-929b-49e5-af0e-9e2524e9d264 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.6 [8,11] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.b [8,11] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.c [2,1] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.1b [2,4] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.0 [5,9] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.1 [4,9] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.c [3,9] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.23 [4,8] 2023-12-11T19:33:22.299 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.3a [4,6] 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/2642090748 expires 2023-12-12T19:24:10.190965+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6801/4268281106 expires 2023-12-12T19:24:10.190965+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/153702681 expires 2023-12-12T19:24:10.190965+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/1199617033 expires 2023-12-12T19:23:41.756564+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/1568045414 expires 2023-12-12T19:24:10.190965+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/294726661 expires 2023-12-12T19:23:41.756564+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6801/3525772822 expires 2023-12-12T19:23:41.756564+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6800/3525772822 expires 2023-12-12T19:23:41.756564+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/1790110256 expires 2023-12-12T19:23:41.756564+0000 2023-12-11T19:33:22.300 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6800/4268281106 expires 2023-12-12T19:24:10.190965+0000 2023-12-11T19:33:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feab00a1500 msgr2=0x7feab00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feab00a1500 0x7feab00a39c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feac8102680 tx=0x7feab8009260 comp rx=0 tx=0).stop 2023-12-11T19:33:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 msgr2=0x7feac8073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:22.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 0x7feac8073f00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feac0007ee0 tx=0x7feac000d3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 shutdown_connections 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feab00a1500 0x7feab00a39c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feac81031c0 0x7feac8073f00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feac8101620 0x7feac8075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 --2- 172.21.15.66:0/612883726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feac8102820 0x7feac80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 >> 172.21.15.66:0/612883726 conn(0x7feac80fcbe0 msgr2=0x7feac8105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 shutdown_connections 2023-12-11T19:33:22.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:22.297+0000 7feacda8d700 1 -- 172.21.15.66:0/612883726 wait complete. 2023-12-11T19:33:22.841 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-11T19:33:22.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:23.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:22 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/612883726' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-11T19:33:23.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:22 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/612883726' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-11T19:33:23.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 -- 172.21.15.66:0/3860171048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 msgr2=0x7ff3c410b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/3860171048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c410b7f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b0a0 tx=0x7ff3bc00b3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 -- 172.21.15.66:0/3860171048 shutdown_connections 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/3860171048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3c4075610 0x7ff3c4073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/3860171048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c410b7f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/3860171048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3c4074030 0x7ff3c40744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 -- 172.21.15.66:0/3860171048 >> 172.21.15.66:0/3860171048 conn(0x7ff3c40fca10 msgr2=0x7ff3c40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 -- 172.21.15.66:0/3860171048 shutdown_connections 2023-12-11T19:33:23.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.063+0000 7ff3c93ff700 1 -- 172.21.15.66:0/3860171048 wait complete. 2023-12-11T19:33:23.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 Processor -- start 2023-12-11T19:33:23.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 -- start start 2023-12-11T19:33:23.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3c4074030 0x7ff3c4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3c4075610 0x7ff3c4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41a1150 con 0x7ff3c4109360 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41a1290 con 0x7ff3c4074030 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c93ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41a1420 con 0x7ff3c4075610 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c41a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57946/0 (socket says 172.21.15.66:57946) 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c37fe700 1 -- 172.21.15.66:0/2396298822 learned_addr learned my addr 172.21.15.66:0/2396298822 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.065+0000 7ff3c27fc700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3c4075610 0x7ff3c4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c2ffd700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3c4074030 0x7ff3c4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c37fe700 1 -- 172.21.15.66:0/2396298822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3c4075610 msgr2=0x7ff3c4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c37fe700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3c4075610 0x7ff3c4199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c37fe700 1 -- 172.21.15.66:0/2396298822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3c4074030 msgr2=0x7ff3c4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c37fe700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3c4074030 0x7ff3c4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c37fe700 1 -- 172.21.15.66:0/2396298822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc009d00 con 0x7ff3c4109360 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c37fe700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c41a0b80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b780 tx=0x7ff3bc00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00e040 con 0x7ff3c4109360 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c41a16a0 con 0x7ff3c4109360 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3bc00be00 con 0x7ff3c4109360 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0052a0 con 0x7ff3c4109360 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.066+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c41a1ac0 con 0x7ff3c4109360 2023-12-11T19:33:23.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.068+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c4108120 con 0x7ff3c4109360 2023-12-11T19:33:23.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.069+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff3bc01d9a0 con 0x7ff3c4109360 2023-12-11T19:33:23.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.071+0000 7ff3abfff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff3ac0a1480 0x7ff3ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.071+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff3bc0cded0 con 0x7ff3c4109360 2023-12-11T19:33:23.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.071+0000 7ff3c2ffd700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff3ac0a1480 0x7ff3ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.072+0000 7ff3c2ffd700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff3ac0a1480 0x7ff3ac0a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009bf0 tx=0x7ff3b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:23.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.075+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff3bc097580 con 0x7ff3c4109360 2023-12-11T19:33:23.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:22 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/612883726' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-11T19:33:23.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.287+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3c4068230 con 0x7ff3c4109360 2023-12-11T19:33:23.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.289+0000 7ff3abfff700 1 -- 172.21.15.66:0/2396298822 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7ff3bc09aba0 con 0x7ff3c4109360 2023-12-11T19:33:23.291 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:23.291 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff3ac0a1480 msgr2=0x7ff3ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff3ac0a1480 0x7ff3ac0a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009bf0 tx=0x7ff3b4009260 comp rx=0 tx=0).stop 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 msgr2=0x7ff3c41a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c41a0b80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00b780 tx=0x7ff3bc00bb60 comp rx=0 tx=0).stop 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 shutdown_connections 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff3ac0a1480 0x7ff3ac0a3940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3c4074030 0x7ff3c4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c4109360 0x7ff3c41a0b80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 --2- 172.21.15.66:0/2396298822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3c4075610 0x7ff3c4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 >> 172.21.15.66:0/2396298822 conn(0x7ff3c40fca10 msgr2=0x7ff3c4105260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:23.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 shutdown_connections 2023-12-11T19:33:23.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.292+0000 7ff3c93ff700 1 -- 172.21.15.66:0/2396298822 wait complete. 2023-12-11T19:33:23.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:33:23.313 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 -- 172.21.15.66:0/2216937276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 msgr2=0x7f320c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 --2- 172.21.15.66:0/2216937276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c0744b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009990 tx=0x7f31fc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 -- 172.21.15.66:0/2216937276 shutdown_connections 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 --2- 172.21.15.66:0/2216937276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 0x7f320c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 --2- 172.21.15.66:0/2216937276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 0x7f320c073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 --2- 172.21.15.66:0/2216937276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c0744b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 -- 172.21.15.66:0/2216937276 >> 172.21.15.66:0/2216937276 conn(0x7f320c0fcb10 msgr2=0x7f320c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 -- 172.21.15.66:0/2216937276 shutdown_connections 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.524+0000 7f3213bff700 1 -- 172.21.15.66:0/2216937276 wait complete. 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.525+0000 7f3213bff700 1 Processor -- start 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.525+0000 7f3213bff700 1 -- start start 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f3213bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f3213bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 0x7f320c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f3213bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 0x7f320c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f3213bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a1280 con 0x7f320c074030 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f3213bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a13c0 con 0x7f320c109460 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f3213bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a1550 con 0x7f320c075610 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57966/0 (socket says 172.21.15.66:57966) 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 -- 172.21.15.66:0/3068586746 learned_addr learned my addr 172.21.15.66:0/3068586746 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321119a700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 0x7f320c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321219c700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 0x7f320c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 -- 172.21.15.66:0/3068586746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 msgr2=0x7f320c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 0x7f320c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 -- 172.21.15.66:0/3068586746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 msgr2=0x7f320c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 0x7f320c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321199b700 1 -- 172.21.15.66:0/3068586746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0095f0 con 0x7f320c074030 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321119a700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 0x7f320c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.526+0000 7f321219c700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 0x7f320c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.527+0000 7f321199b700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c1997a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f320800d950 tx=0x7f320800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.527+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208009b40 con 0x7f320c074030 2023-12-11T19:33:23.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.527+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1a1830 con 0x7f320c074030 2023-12-11T19:33:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.527+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3208009ca0 con 0x7f320c074030 2023-12-11T19:33:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.527+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320801b650 con 0x7f320c074030 2023-12-11T19:33:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.527+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1a1d40 con 0x7f320c074030 2023-12-11T19:33:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.528+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c108220 con 0x7f320c074030 2023-12-11T19:33:23.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.530+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3208010460 con 0x7f320c074030 2023-12-11T19:33:23.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.531+0000 7f3202ffd700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31f80a1480 0x7f31f80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.531+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32080cd0c0 con 0x7f320c074030 2023-12-11T19:33:23.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.532+0000 7f321119a700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31f80a1480 0x7f31f80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.532+0000 7f321119a700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31f80a1480 0x7f31f80a3940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f31fc003780 tx=0x7f31fc003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:23.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.535+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f32080966e0 con 0x7f320c074030 2023-12-11T19:33:23.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.752+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f320c107c00 con 0x7f320c074030 2023-12-11T19:33:23.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.753+0000 7f3202ffd700 1 -- 172.21.15.66:0/3068586746 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3208099d00 con 0x7f320c074030 2023-12-11T19:33:23.755 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:33:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31f80a1480 msgr2=0x7f31f80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31f80a1480 0x7f31f80a3940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f31fc003780 tx=0x7f31fc003680 comp rx=0 tx=0).stop 2023-12-11T19:33:23.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 msgr2=0x7f320c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c1997a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f320800d950 tx=0x7f320800dd10 comp rx=0 tx=0).stop 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 shutdown_connections 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31f80a1480 0x7f31f80a3940 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c109460 0x7f320c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c074030 0x7f320c1997a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 --2- 172.21.15.66:0/3068586746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f320c075610 0x7f320c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 >> 172.21.15.66:0/3068586746 conn(0x7f320c0fcb10 msgr2=0x7f320c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 shutdown_connections 2023-12-11T19:33:23.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.756+0000 7f3213bff700 1 -- 172.21.15.66:0/3068586746 wait complete. 2023-12-11T19:33:23.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3445910703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f01020a0 msgr2=0x7fb7f0104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3445910703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f01020a0 0x7fb7f0104490 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009990 tx=0x7fb7e0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3445910703 shutdown_connections 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3445910703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7f0107300 0x7fb7f01096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3445910703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f01020a0 0x7fb7f0104490 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3445910703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f01049d0 0x7fb7f0106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3445910703 >> 172.21.15.66:0/3445910703 conn(0x7fb7f00fb9f0 msgr2=0x7fb7f00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3445910703 shutdown_connections 2023-12-11T19:33:23.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.986+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3445910703 wait complete. 2023-12-11T19:33:23.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.987+0000 7fb7f65ff700 1 Processor -- start 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 -- start start 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f01020a0 0x7fb7f01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7f01049d0 0x7fb7f0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 0x7fb7f019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f019ef10 con 0x7fb7f0107300 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f019f050 con 0x7fb7f01020a0 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f65ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f019f1e0 con 0x7fb7f01049d0 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f4b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 0x7fb7f019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f4b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 0x7fb7f019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57984/0 (socket says 172.21.15.66:57984) 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f4b9c700 1 -- 172.21.15.66:0/3139613629 learned_addr learned my addr 172.21.15.66:0/3139613629 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7effff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f01020a0 0x7fb7f01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.988+0000 7fb7f4b9c700 1 -- 172.21.15.66:0/3139613629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7f01049d0 msgr2=0x7fb7f0197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7ef7fe700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7f01049d0 0x7fb7f0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f4b9c700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7f01049d0 0x7fb7f0197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f4b9c700 1 -- 172.21.15.66:0/3139613629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f01020a0 msgr2=0x7fb7f01973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f4b9c700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f01020a0 0x7fb7f01973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f4b9c700 1 -- 172.21.15.66:0/3139613629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00095f0 con 0x7fb7f0107300 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f4b9c700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 0x7fb7f019e940 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800ef30 tx=0x7fb7e800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8018070 con 0x7fb7f0107300 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7e800f040 con 0x7fb7f0107300 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f019f460 con 0x7fb7f0107300 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8003d00 con 0x7fb7f0107300 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.989+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f019f920 con 0x7fb7f0107300 2023-12-11T19:33:23.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.990+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f00fd570 con 0x7fb7f0107300 2023-12-11T19:33:23.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.992+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb7e800caa0 con 0x7fb7f0107300 2023-12-11T19:33:23.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.994+0000 7fb7ed7fa700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb7d80a1480 0x7fb7d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:23.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.994+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb7e80cca90 con 0x7fb7f0107300 2023-12-11T19:33:23.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.994+0000 7fb7effff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb7d80a1480 0x7fb7d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:23.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.995+0000 7fb7effff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb7d80a1480 0x7fb7d80a3940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b2e0 tx=0x7fb7e0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:23.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:23.998+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb7e8096140 con 0x7fb7f0107300 2023-12-11T19:33:24.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:23 smithi066 ceph-mon[97920]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-11T19:33:24.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:23 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2396298822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:24.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:23 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3068586746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:24.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:23 smithi070 ceph-mon[104044]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-11T19:33:24.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:23 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2396298822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:24.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:23 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3068586746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:24.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.213+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7f00fcf50 con 0x7fb7f0107300 2023-12-11T19:33:24.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.214+0000 7fb7ed7fa700 1 -- 172.21.15.66:0/3139613629 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7e8099760 con 0x7fb7f0107300 2023-12-11T19:33:24.215 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb7d80a1480 msgr2=0x7fb7d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb7d80a1480 0x7fb7d80a3940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b2e0 tx=0x7fb7e0005fd0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 msgr2=0x7fb7f019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 0x7fb7f019e940 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800ef30 tx=0x7fb7e800c450 comp rx=0 tx=0).stop 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 shutdown_connections 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb7d80a1480 0x7fb7d80a3940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f01020a0 0x7fb7f01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb7f0107300 0x7fb7f019e940 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 --2- 172.21.15.66:0/3139613629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7f01049d0 0x7fb7f0197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 >> 172.21.15.66:0/3139613629 conn(0x7fb7f00fb9f0 msgr2=0x7fb7f0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.216+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 shutdown_connections 2023-12-11T19:33:24.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.217+0000 7fb7f65ff700 1 -- 172.21.15.66:0/3139613629 wait complete. 2023-12-11T19:33:24.237 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:33:24.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:23 smithi167 ceph-mon[102750]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-11T19:33:24.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:23 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2396298822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:24.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:23 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3068586746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:24.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 -- 172.21.15.66:0/236943109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c103130 msgr2=0x7f1f5c1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 --2- 172.21.15.66:0/236943109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c103130 0x7f1f5c1099e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400b0d0 tx=0x7f1f5400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 -- 172.21.15.66:0/236943109 shutdown_connections 2023-12-11T19:33:24.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 --2- 172.21.15.66:0/236943109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 --2- 172.21.15.66:0/236943109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c103130 0x7f1f5c1099e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 --2- 172.21.15.66:0/236943109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f5c102790 0x7f1f5c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.446+0000 7f1f61dff700 1 -- 172.21.15.66:0/236943109 >> 172.21.15.66:0/236943109 conn(0x7f1f5c0fcb10 msgr2=0x7f1f5c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:24.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.447+0000 7f1f61dff700 1 -- 172.21.15.66:0/236943109 shutdown_connections 2023-12-11T19:33:24.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.447+0000 7f1f61dff700 1 -- 172.21.15.66:0/236943109 wait complete. 2023-12-11T19:33:24.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 Processor -- start 2023-12-11T19:33:24.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 -- start start 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f5c102790 0x7f1f5c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f5c103130 0x7f1f5c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1a1270 con 0x7f1f5c1015a0 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1a13b0 con 0x7f1f5c102790 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f61dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1a1540 con 0x7f1f5c103130 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f5b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f5b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58002/0 (socket says 172.21.15.66:58002) 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f5b7fe700 1 -- 172.21.15.66:0/3664954090 learned_addr learned my addr 172.21.15.66:0/3664954090 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:24.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.448+0000 7f1f5affd700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f5c102790 0x7f1f5c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5bfff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f5c103130 0x7f1f5c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5b7fe700 1 -- 172.21.15.66:0/3664954090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f5c103130 msgr2=0x7f1f5c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5b7fe700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f5c103130 0x7f1f5c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5b7fe700 1 -- 172.21.15.66:0/3664954090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f5c102790 msgr2=0x7f1f5c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5b7fe700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f5c102790 0x7f1f5c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5b7fe700 1 -- 172.21.15.66:0/3664954090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54009d00 con 0x7f1f5c1015a0 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f5b7fe700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c199740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00e970 tx=0x7f1f4c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c009790 con 0x7f1f5c1015a0 2023-12-11T19:33:24.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f4c00eec0 con 0x7f1f5c1015a0 2023-12-11T19:33:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c1a1820 con 0x7f1f5c1015a0 2023-12-11T19:33:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c003c10 con 0x7f1f5c1015a0 2023-12-11T19:33:24.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.449+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c1a1d80 con 0x7f1f5c1015a0 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.450+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c068310 con 0x7f1f5c1015a0 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.452+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1f4c003db0 con 0x7f1f5c1015a0 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.454+0000 7f1f58ff9700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1f440a1500 0x7f1f440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.454+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f4c014070 con 0x7f1f5c1015a0 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.454+0000 7f1f5affd700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1f440a1500 0x7f1f440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.454+0000 7f1f5affd700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1f440a1500 0x7f1f440a39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f50007be0 tx=0x7f1f500073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:24.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.457+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1f4c096560 con 0x7f1f5c1015a0 2023-12-11T19:33:24.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.670+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f5c10cee0 con 0x7f1f5c1015a0 2023-12-11T19:33:24.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.670+0000 7f1f58ff9700 1 -- 172.21.15.66:0/3664954090 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f4c099b80 con 0x7f1f5c1015a0 2023-12-11T19:33:24.672 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:33:24.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1f440a1500 msgr2=0x7f1f440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1f440a1500 0x7f1f440a39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f50007be0 tx=0x7f1f500073d0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 msgr2=0x7f1f5c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c199740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00e970 tx=0x7f1f4c00ec80 comp rx=0 tx=0).stop 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 shutdown_connections 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1f440a1500 0x7f1f440a39c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f5c103130 0x7f1f5c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f5c1015a0 0x7f1f5c199740 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 --2- 172.21.15.66:0/3664954090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f5c102790 0x7f1f5c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 >> 172.21.15.66:0/3664954090 conn(0x7f1f5c0fcb10 msgr2=0x7f1f5c107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 shutdown_connections 2023-12-11T19:33:24.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.673+0000 7f1f61dff700 1 -- 172.21.15.66:0/3664954090 wait complete. 2023-12-11T19:33:24.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:33:24.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 -- 172.21.15.66:0/407306834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 msgr2=0x7ff554102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/407306834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554102af0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5440099c0 tx=0x7ff544009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 -- 172.21.15.66:0/407306834 shutdown_connections 2023-12-11T19:33:24.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/407306834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff554103030 0x7ff5541098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/407306834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 0x7ff5541018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/407306834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554102af0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 -- 172.21.15.66:0/407306834 >> 172.21.15.66:0/407306834 conn(0x7ff5540fca10 msgr2=0x7ff5540fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:24.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 -- 172.21.15.66:0/407306834 shutdown_connections 2023-12-11T19:33:24.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.902+0000 7ff55b9ff700 1 -- 172.21.15.66:0/407306834 wait complete. 2023-12-11T19:33:24.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 Processor -- start 2023-12-11T19:33:24.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 -- start start 2023-12-11T19:33:24.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 0x7ff5541996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff554103030 0x7ff5541a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541a1180 con 0x7ff554102690 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541a12c0 con 0x7ff554103030 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff55b9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541a1450 con 0x7ff5541014a0 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff558f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff558f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58022/0 (socket says 172.21.15.66:58022) 2023-12-11T19:33:24.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.904+0000 7ff558f9a700 1 -- 172.21.15.66:0/3731829316 learned_addr learned my addr 172.21.15.66:0/3731829316 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:24.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff559f9c700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff554103030 0x7ff5541a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff55979b700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 0x7ff5541996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff558f9a700 1 -- 172.21.15.66:0/3731829316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 msgr2=0x7ff5541996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff558f9a700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 0x7ff5541996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff558f9a700 1 -- 172.21.15.66:0/3731829316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff554103030 msgr2=0x7ff5541a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:24.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff558f9a700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff554103030 0x7ff5541a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:24.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff558f9a700 1 -- 172.21.15.66:0/3731829316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440095f0 con 0x7ff554102690 2023-12-11T19:33:24.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff55979b700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 0x7ff5541996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:24.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff558f9a700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554199be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff544000c00 tx=0x7ff544003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:24.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401d070 con 0x7ff554102690 2023-12-11T19:33:24.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff54400bd30 con 0x7ff554102690 2023-12-11T19:33:24.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544022da0 con 0x7ff554102690 2023-12-11T19:33:24.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.905+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5541a16d0 con 0x7ff554102690 2023-12-11T19:33:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.906+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5541a1b20 con 0x7ff554102690 2023-12-11T19:33:24.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.907+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554068230 con 0x7ff554102690 2023-12-11T19:33:24.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.908+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff544003fd0 con 0x7ff554102690 2023-12-11T19:33:24.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.909+0000 7ff54a7fc700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff5400a1480 0x7ff5400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:24.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.909+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff5440ce3f0 con 0x7ff554102690 2023-12-11T19:33:24.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.909+0000 7ff55979b700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff5400a1480 0x7ff5400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:24.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.911+0000 7ff55979b700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff5400a1480 0x7ff5400a3940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5541024f0 tx=0x7ff550008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:24.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:24.914+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff544097bc0 con 0x7ff554102690 2023-12-11T19:33:25.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:24 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3139613629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:25.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:24 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3664954090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:25.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:24 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3139613629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:25.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:24 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3664954090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:25.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.175+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff55410ccd0 con 0x7ff554102690 2023-12-11T19:33:25.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.175+0000 7ff54a7fc700 1 -- 172.21.15.66:0/3731829316 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7ff544027020 con 0x7ff554102690 2023-12-11T19:33:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:25.176 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:33:25.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff5400a1480 msgr2=0x7ff5400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff5400a1480 0x7ff5400a3940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5541024f0 tx=0x7ff550008040 comp rx=0 tx=0).stop 2023-12-11T19:33:25.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 msgr2=0x7ff554199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554199be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff544000c00 tx=0x7ff544003a50 comp rx=0 tx=0).stop 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 shutdown_connections 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff5400a1480 0x7ff5400a3940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff554103030 0x7ff5541a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5541014a0 0x7ff5541996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 --2- 172.21.15.66:0/3731829316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff554102690 0x7ff554199be0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 >> 172.21.15.66:0/3731829316 conn(0x7ff5540fca10 msgr2=0x7ff554073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 shutdown_connections 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.178+0000 7ff55b9ff700 1 -- 172.21.15.66:0/3731829316 wait complete. 2023-12-11T19:33:25.179 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:33:25.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:25.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:24 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3139613629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:25.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:24 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3664954090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 -- 172.21.15.66:0/1165287119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a001014a0 msgr2=0x7f1a001018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1165287119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a001014a0 0x7f1a001018c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0099c0 tx=0x7f19fc009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 -- 172.21.15.66:0/1165287119 shutdown_connections 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1165287119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a00103030 0x7f1a001098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1165287119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a001014a0 0x7f1a001018c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1165287119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a00102690 0x7f1a00102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 -- 172.21.15.66:0/1165287119 >> 172.21.15.66:0/1165287119 conn(0x7f1a000fca10 msgr2=0x7f1a000fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:25.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.405+0000 7f1a07bff700 1 -- 172.21.15.66:0/1165287119 shutdown_connections 2023-12-11T19:33:25.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.406+0000 7f1a07bff700 1 -- 172.21.15.66:0/1165287119 wait complete. 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.406+0000 7f1a07bff700 1 Processor -- start 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 -- start start 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a001014a0 0x7f1a00197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 0x7f1a001978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a00103030 0x7f1a0019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0019ee90 con 0x7f1a00102690 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0019efd0 con 0x7f1a001014a0 2023-12-11T19:33:25.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a07bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0019f160 con 0x7f1a00103030 2023-12-11T19:33:25.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a0519a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 0x7f1a001978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a0519a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 0x7f1a001978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58032/0 (socket says 172.21.15.66:58032) 2023-12-11T19:33:25.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a0519a700 1 -- 172.21.15.66:0/1005205096 learned_addr learned my addr 172.21.15.66:0/1005205096 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:25.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.407+0000 7f1a0619c700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a00103030 0x7f1a0019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0599b700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a001014a0 0x7f1a00197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0519a700 1 -- 172.21.15.66:0/1005205096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a00103030 msgr2=0x7f1a0019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0519a700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a00103030 0x7f1a0019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0519a700 1 -- 172.21.15.66:0/1005205096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a001014a0 msgr2=0x7f1a00197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0519a700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a001014a0 0x7f1a00197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0519a700 1 -- 172.21.15.66:0/1005205096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0095f0 con 0x7f1a00102690 2023-12-11T19:33:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0599b700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a001014a0 0x7f1a00197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a0519a700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 0x7f1a001978a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d950 tx=0x7f19f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0009940 con 0x7f1a00102690 2023-12-11T19:33:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0019f440 con 0x7f1a00102690 2023-12-11T19:33:25.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19f0010460 con 0x7f1a00102690 2023-12-11T19:33:25.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f000f620 con 0x7f1a00102690 2023-12-11T19:33:25.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.408+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0019f950 con 0x7f1a00102690 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.409+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a00108910 con 0x7f1a00102690 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.411+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f19f00105d0 con 0x7f1a00102690 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.413+0000 7f19f6ffd700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19ec0a1480 0x7f19ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.413+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f19f00ccb90 con 0x7f1a00102690 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.413+0000 7f1a0599b700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19ec0a1480 0x7f19ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.414+0000 7f1a0599b700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19ec0a1480 0x7f19ec0a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006010 tx=0x7f19fc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:25.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.415+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f19f00961b0 con 0x7f1a00102690 2023-12-11T19:33:25.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.631+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a0010cbb0 con 0x7f1a00102690 2023-12-11T19:33:25.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.632+0000 7f19f6ffd700 1 -- 172.21.15.66:0/1005205096 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f19f0016090 con 0x7f1a00102690 2023-12-11T19:33:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:25.633 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:25.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19ec0a1480 msgr2=0x7f19ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19ec0a1480 0x7f19ec0a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006010 tx=0x7f19fc00b540 comp rx=0 tx=0).stop 2023-12-11T19:33:25.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 msgr2=0x7f1a001978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 0x7f1a001978a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d950 tx=0x7f19f000dc60 comp rx=0 tx=0).stop 2023-12-11T19:33:25.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 shutdown_connections 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f19ec0a1480 0x7f19ec0a3940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a001014a0 0x7f1a00197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a00103030 0x7f1a0019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 --2- 172.21.15.66:0/1005205096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a00102690 0x7f1a001978a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 >> 172.21.15.66:0/1005205096 conn(0x7f1a000fca10 msgr2=0x7f1a00105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 shutdown_connections 2023-12-11T19:33:25.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.635+0000 7f1a07bff700 1 -- 172.21.15.66:0/1005205096 wait complete. 2023-12-11T19:33:25.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:33:25.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:33:25.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 -- 172.21.15.66:0/2617069417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d01015a0 msgr2=0x7f31d01019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 --2- 172.21.15.66:0/2617069417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d01015a0 0x7f31d01019c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f31c00099c0 tx=0x7f31c0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 -- 172.21.15.66:0/2617069417 shutdown_connections 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 --2- 172.21.15.66:0/2617069417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d0103130 0x7f31d01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 --2- 172.21.15.66:0/2617069417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d01015a0 0x7f31d01019c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 --2- 172.21.15.66:0/2617069417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31d0102790 0x7f31d0102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 -- 172.21.15.66:0/2617069417 >> 172.21.15.66:0/2617069417 conn(0x7f31d00fcb10 msgr2=0x7f31d00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 -- 172.21.15.66:0/2617069417 shutdown_connections 2023-12-11T19:33:25.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.874+0000 7f31d55ff700 1 -- 172.21.15.66:0/2617069417 wait complete. 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.875+0000 7f31d55ff700 1 Processor -- start 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.875+0000 7f31d55ff700 1 -- start start 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.875+0000 7f31d55ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31d01015a0 0x7f31d0197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.875+0000 7f31d55ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d0102790 0x7f31d0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.875+0000 7f31d55ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 0x7f31d019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31d55ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019f040 con 0x7f31d0103130 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31d55ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019f180 con 0x7f31d0102790 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31d55ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019f310 con 0x7f31d01015a0 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 0x7f31d019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 0x7f31d019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58050/0 (socket says 172.21.15.66:58050) 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 -- 172.21.15.66:0/1610216555 learned_addr learned my addr 172.21.15.66:0/1610216555 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31c7fff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d0102790 0x7f31d0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 -- 172.21.15.66:0/1610216555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31d01015a0 msgr2=0x7f31d0197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31d01015a0 0x7f31d0197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 -- 172.21.15.66:0/1610216555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d0102790 msgr2=0x7f31d0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d0102790 0x7f31d0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 -- 172.21.15.66:0/1610216555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00095f0 con 0x7f31d0103130 2023-12-11T19:33:25.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.876+0000 7f31cf7fe700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 0x7f31d019ea70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f31c8007ee0 tx=0x7f31c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:25.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.877+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8017070 con 0x7f31d0103130 2023-12-11T19:33:25.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.877+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c800f040 con 0x7f31d0103130 2023-12-11T19:33:25.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.877+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d019f5f0 con 0x7f31d0103130 2023-12-11T19:33:25.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.877+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8005710 con 0x7f31d0103130 2023-12-11T19:33:25.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.877+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d019fab0 con 0x7f31d0103130 2023-12-11T19:33:25.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.878+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d0108a90 con 0x7f31d0103130 2023-12-11T19:33:25.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.880+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f31c8005870 con 0x7f31d0103130 2023-12-11T19:33:25.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.881+0000 7f31ccff9700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31bc0a1480 0x7f31bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:25.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.881+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31c80cc5f0 con 0x7f31d0103130 2023-12-11T19:33:25.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.881+0000 7f31ceffd700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31bc0a1480 0x7f31bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:25.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.882+0000 7f31ceffd700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31bc0a1480 0x7f31bc0a3940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f31c00098e0 tx=0x7f31c0009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:25.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:25.885+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f31c8095c10 con 0x7f31d0103130 2023-12-11T19:33:26.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:25 smithi066 ceph-mon[97920]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 10 op/s 2023-12-11T19:33:26.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:25 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3731829316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:26.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:25 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1005205096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:26.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.100+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31d010cf70 con 0x7f31d0103130 2023-12-11T19:33:26.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.101+0000 7f31ccff9700 1 -- 172.21.15.66:0/1610216555 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31c8099230 con 0x7f31d0103130 2023-12-11T19:33:26.101 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:33:26.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31bc0a1480 msgr2=0x7f31bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31bc0a1480 0x7f31bc0a3940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f31c00098e0 tx=0x7f31c0009f10 comp rx=0 tx=0).stop 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 msgr2=0x7f31d019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 0x7f31d019ea70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f31c8007ee0 tx=0x7f31c800d3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 shutdown_connections 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f31bc0a1480 0x7f31bc0a3940 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d0103130 0x7f31d019ea70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31d01015a0 0x7f31d0197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 --2- 172.21.15.66:0/1610216555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d0102790 0x7f31d0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 >> 172.21.15.66:0/1610216555 conn(0x7f31d00fcb10 msgr2=0x7f31d0105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 shutdown_connections 2023-12-11T19:33:26.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.103+0000 7f31d55ff700 1 -- 172.21.15.66:0/1610216555 wait complete. 2023-12-11T19:33:26.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:25 smithi070 ceph-mon[104044]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 10 op/s 2023-12-11T19:33:26.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:25 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3731829316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:26.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:25 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1005205096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:26.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:33:26.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:25 smithi167 ceph-mon[102750]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 10 op/s 2023-12-11T19:33:26.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:25 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3731829316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:26.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:25 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1005205096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.335+0000 7f3271dff700 1 -- 172.21.15.66:0/498657380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c102790 msgr2=0x7f326c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.335+0000 7f3271dff700 1 --2- 172.21.15.66:0/498657380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c102790 0x7f326c102bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f32600099c0 tx=0x7f3260009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 -- 172.21.15.66:0/498657380 shutdown_connections 2023-12-11T19:33:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 --2- 172.21.15.66:0/498657380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f326c1015a0 0x7f326c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 --2- 172.21.15.66:0/498657380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 0x7f326c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 --2- 172.21.15.66:0/498657380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c102790 0x7f326c102bf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 -- 172.21.15.66:0/498657380 >> 172.21.15.66:0/498657380 conn(0x7f326c0fcb10 msgr2=0x7f326c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 -- 172.21.15.66:0/498657380 shutdown_connections 2023-12-11T19:33:26.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.336+0000 7f3271dff700 1 -- 172.21.15.66:0/498657380 wait complete. 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 Processor -- start 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 -- start start 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 0x7f326c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f326c102790 0x7f326c199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 0x7f326c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1a1190 con 0x7f326c1015a0 2023-12-11T19:33:26.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1a12d0 con 0x7f326c102790 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f3271dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1a1460 con 0x7f326c103130 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.337+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 0x7f326c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 0x7f326c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58070/0 (socket says 172.21.15.66:58070) 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 -- 172.21.15.66:0/3798083552 learned_addr learned my addr 172.21.15.66:0/3798083552 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326affd700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f326c102790 0x7f326c199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326bfff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 0x7f326c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 -- 172.21.15.66:0/3798083552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 msgr2=0x7f326c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 0x7f326c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 -- 172.21.15.66:0/3798083552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f326c102790 msgr2=0x7f326c199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f326c102790 0x7f326c199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 -- 172.21.15.66:0/3798083552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600095f0 con 0x7f326c1015a0 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326bfff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 0x7f326c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f326b7fe700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 0x7f326c1996b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e970 tx=0x7f325c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:26.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.338+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c009790 con 0x7f326c1015a0 2023-12-11T19:33:26.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.339+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c1a1740 con 0x7f326c1015a0 2023-12-11T19:33:26.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.339+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f325c00eec0 con 0x7f326c1015a0 2023-12-11T19:33:26.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.339+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c003c10 con 0x7f326c1015a0 2023-12-11T19:33:26.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.339+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c1a1c50 con 0x7f326c1015a0 2023-12-11T19:33:26.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.340+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c068310 con 0x7f326c1015a0 2023-12-11T19:33:26.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.342+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f325c003db0 con 0x7f326c1015a0 2023-12-11T19:33:26.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.343+0000 7f3268ff9700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f32540a1480 0x7f32540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.343+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f325c014070 con 0x7f326c1015a0 2023-12-11T19:33:26.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.343+0000 7f326affd700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f32540a1480 0x7f32540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.344+0000 7f326affd700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f32540a1480 0x7f32540a3940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3260009990 tx=0x7f3260003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:26.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.347+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f325c0965c0 con 0x7f326c1015a0 2023-12-11T19:33:26.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.561+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f326c1a1fd0 con 0x7f326c1015a0 2023-12-11T19:33:26.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.561+0000 7f3268ff9700 1 -- 172.21.15.66:0/3798083552 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f325c099be0 con 0x7f326c1015a0 2023-12-11T19:33:26.562 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f32540a1480 msgr2=0x7f32540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f32540a1480 0x7f32540a3940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3260009990 tx=0x7f3260003680 comp rx=0 tx=0).stop 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 msgr2=0x7f326c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 0x7f326c1996b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e970 tx=0x7f325c00ec80 comp rx=0 tx=0).stop 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 shutdown_connections 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f32540a1480 0x7f32540a3940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f326c103130 0x7f326c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f326c1015a0 0x7f326c1996b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 --2- 172.21.15.66:0/3798083552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f326c102790 0x7f326c199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 >> 172.21.15.66:0/3798083552 conn(0x7f326c0fcb10 msgr2=0x7f326c107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 shutdown_connections 2023-12-11T19:33:26.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.564+0000 7f3271dff700 1 -- 172.21.15.66:0/3798083552 wait complete. 2023-12-11T19:33:26.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 -- 172.21.15.66:0/1804676768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11141015a0 msgr2=0x7f11141019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 --2- 172.21.15.66:0/1804676768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11141015a0 0x7f11141019c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11040099c0 tx=0x7f1104009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 -- 172.21.15.66:0/1804676768 shutdown_connections 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 --2- 172.21.15.66:0/1804676768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1114103130 0x7f11141099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 --2- 172.21.15.66:0/1804676768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11141015a0 0x7f11141019c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 --2- 172.21.15.66:0/1804676768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114102790 0x7f1114102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.795+0000 7f11195ff700 1 -- 172.21.15.66:0/1804676768 >> 172.21.15.66:0/1804676768 conn(0x7f11140fcb10 msgr2=0x7f11140fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.796+0000 7f11195ff700 1 -- 172.21.15.66:0/1804676768 shutdown_connections 2023-12-11T19:33:26.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.796+0000 7f11195ff700 1 -- 172.21.15.66:0/1804676768 wait complete. 2023-12-11T19:33:26.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.797+0000 7f11195ff700 1 Processor -- start 2023-12-11T19:33:26.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.797+0000 7f11195ff700 1 -- start start 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11195ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141015a0 0x7f1114199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11195ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 0x7f1114199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11195ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114103130 0x7f11141a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11195ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11141a1230 con 0x7f1114102790 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11195ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11141a1370 con 0x7f1114103130 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11195ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11141a1500 con 0x7f11141015a0 2023-12-11T19:33:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 0x7f1114199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 0x7f1114199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58096/0 (socket says 172.21.15.66:58096) 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 -- 172.21.15.66:0/2499792784 learned_addr learned my addr 172.21.15.66:0/2499792784 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f1112ffd700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141015a0 0x7f1114199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 -- 172.21.15.66:0/2499792784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141015a0 msgr2=0x7f1114199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11137fe700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114103130 0x7f11141a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141015a0 0x7f1114199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 -- 172.21.15.66:0/2499792784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114103130 msgr2=0x7f11141a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114103130 0x7f11141a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.798+0000 7f11127fc700 1 -- 172.21.15.66:0/2499792784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040095f0 con 0x7f1114102790 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f1112ffd700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141015a0 0x7f1114199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f11137fe700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114103130 0x7f11141a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f11127fc700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 0x7f1114199c90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f110800e9a0 tx=0x7f110800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108009790 con 0x7f1114102790 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f110800cbd0 con 0x7f1114102790 2023-12-11T19:33:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11141a17e0 con 0x7f1114102790 2023-12-11T19:33:26.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108018680 con 0x7f1114102790 2023-12-11T19:33:26.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.799+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11141a1cf0 con 0x7f1114102790 2023-12-11T19:33:26.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.800+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1114068310 con 0x7f1114102790 2023-12-11T19:33:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.802+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f110800cd40 con 0x7f1114102790 2023-12-11T19:33:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.804+0000 7f10fbfff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f10fc0a1480 0x7f10fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.804+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1108014070 con 0x7f1114102790 2023-12-11T19:33:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.804+0000 7f1112ffd700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f10fc0a1480 0x7f10fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.805+0000 7f1112ffd700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f10fc0a1480 0x7f10fc0a3940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1104009990 tx=0x7f110401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:26.808+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f11080522f0 con 0x7f1114102790 2023-12-11T19:33:27.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.023+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11141a2010 con 0x7f1114102790 2023-12-11T19:33:27.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.024+0000 7f10fbfff700 1 -- 172.21.15.66:0/2499792784 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1108096660 con 0x7f1114102790 2023-12-11T19:33:27.024 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.026+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f10fc0a1480 msgr2=0x7f10fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.026+0000 7f11195ff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f10fc0a1480 0x7f10fc0a3940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1104009990 tx=0x7f110401a040 comp rx=0 tx=0).stop 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 msgr2=0x7f1114199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 0x7f1114199c90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f110800e9a0 tx=0x7f110800ed60 comp rx=0 tx=0).stop 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 shutdown_connections 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f10fc0a1480 0x7f10fc0a3940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1114103130 0x7f11141a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141015a0 0x7f1114199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 --2- 172.21.15.66:0/2499792784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1114102790 0x7f1114199c90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 >> 172.21.15.66:0/2499792784 conn(0x7f11140fcb10 msgr2=0x7f1114107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 shutdown_connections 2023-12-11T19:33:27.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.027+0000 7f11195ff700 1 -- 172.21.15.66:0/2499792784 wait complete. 2023-12-11T19:33:27.044 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:26 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1610216555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.044 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:26 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3798083552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.046 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:33:27.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:26 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1610216555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:26 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3798083552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 -- 172.21.15.66:0/1900447209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42301036b0 msgr2=0x7f4230103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 --2- 172.21.15.66:0/1900447209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42301036b0 0x7f4230103ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f42200099c0 tx=0x7f4220009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 -- 172.21.15.66:0/1900447209 shutdown_connections 2023-12-11T19:33:27.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 --2- 172.21.15.66:0/1900447209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4230105240 0x7f423010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 --2- 172.21.15.66:0/1900447209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42301036b0 0x7f4230103ad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 --2- 172.21.15.66:0/1900447209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301048a0 0x7f4230104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 -- 172.21.15.66:0/1900447209 >> 172.21.15.66:0/1900447209 conn(0x7f42300fec40 msgr2=0x7f4230101060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:27.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 -- 172.21.15.66:0/1900447209 shutdown_connections 2023-12-11T19:33:27.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.254+0000 7f42365ff700 1 -- 172.21.15.66:0/1900447209 wait complete. 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.255+0000 7f42365ff700 1 Processor -- start 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.255+0000 7f42365ff700 1 -- start start 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f42365ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301036b0 0x7f4230199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f42365ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42301048a0 0x7f4230199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f42365ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 0x7f42301a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f42365ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301a10c0 con 0x7f4230105240 2023-12-11T19:33:27.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f42365ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301a1200 con 0x7f42301048a0 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f42365ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301a1390 con 0x7f42301036b0 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f4234b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 0x7f42301a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f422ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301036b0 0x7f4230199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f4234b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 0x7f42301a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58114/0 (socket says 172.21.15.66:58114) 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f4234b9c700 1 -- 172.21.15.66:0/820036813 learned_addr learned my addr 172.21.15.66:0/820036813 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.256+0000 7f422f7fe700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42301048a0 0x7f4230199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f4234b9c700 1 -- 172.21.15.66:0/820036813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301036b0 msgr2=0x7f4230199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f4234b9c700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301036b0 0x7f4230199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f4234b9c700 1 -- 172.21.15.66:0/820036813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42301048a0 msgr2=0x7f4230199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f4234b9c700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42301048a0 0x7f4230199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f4234b9c700 1 -- 172.21.15.66:0/820036813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200095f0 con 0x7f4230105240 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f422ffff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301036b0 0x7f4230199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f4234b9c700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 0x7f42301a0af0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4228007ee0 tx=0x7f422800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228017070 con 0x7f4230105240 2023-12-11T19:33:27.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f422800f040 con 0x7f4230105240 2023-12-11T19:33:27.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4230075a70 con 0x7f4230105240 2023-12-11T19:33:27.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228005570 con 0x7f4230105240 2023-12-11T19:33:27.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.257+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4230075f30 con 0x7f4230105240 2023-12-11T19:33:27.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.259+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423010abd0 con 0x7f4230105240 2023-12-11T19:33:27.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.261+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f422800db20 con 0x7f4230105240 2023-12-11T19:33:27.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.262+0000 7f422d7fa700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42180a1480 0x7f42180a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.262+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42280cc4d0 con 0x7f4230105240 2023-12-11T19:33:27.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.262+0000 7f422ffff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42180a1480 0x7f42180a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.263+0000 7f422ffff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42180a1480 0x7f42180a3940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4220009990 tx=0x7f422001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:27.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.266+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4228095af0 con 0x7f4230105240 2023-12-11T19:33:27.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:26 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1610216555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:26 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3798083552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.528+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4230068230 con 0x7f4230105240 2023-12-11T19:33:27.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.529+0000 7f422d7fa700 1 -- 172.21.15.66:0/820036813 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f4228099110 con 0x7f4230105240 2023-12-11T19:33:27.530 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:27.530 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:33:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42180a1480 msgr2=0x7f42180a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42180a1480 0x7f42180a3940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4220009990 tx=0x7f422001a040 comp rx=0 tx=0).stop 2023-12-11T19:33:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 msgr2=0x7f42301a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 0x7f42301a0af0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4228007ee0 tx=0x7f422800d3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 shutdown_connections 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42180a1480 0x7f42180a3940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4230105240 0x7f42301a0af0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42301036b0 0x7f4230199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 --2- 172.21.15.66:0/820036813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42301048a0 0x7f4230199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 >> 172.21.15.66:0/820036813 conn(0x7f42300fec40 msgr2=0x7f4230107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 shutdown_connections 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.532+0000 7f42365ff700 1 -- 172.21.15.66:0/820036813 wait complete. 2023-12-11T19:33:27.533 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:33:27.551 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 -- 172.21.15.66:0/3167586051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad01014a0 msgr2=0x7f9ad01018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/3167586051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad01014a0 0x7f9ad01018c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0099c0 tx=0x7f9acc009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 -- 172.21.15.66:0/3167586051 shutdown_connections 2023-12-11T19:33:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/3167586051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0103030 0x7f9ad01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/3167586051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad01014a0 0x7f9ad01018c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/3167586051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad0102690 0x7f9ad0102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 -- 172.21.15.66:0/3167586051 >> 172.21.15.66:0/3167586051 conn(0x7f9ad00fca10 msgr2=0x7f9ad00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 -- 172.21.15.66:0/3167586051 shutdown_connections 2023-12-11T19:33:27.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.759+0000 7f9ad77ff700 1 -- 172.21.15.66:0/3167586051 wait complete. 2023-12-11T19:33:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 Processor -- start 2023-12-11T19:33:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 -- start start 2023-12-11T19:33:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01014a0 0x7f9ad0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0102690 0x7f9ad00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad0103030 0x7f9ad0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad00746a0 con 0x7f9ad0103030 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad0074810 con 0x7f9ad01014a0 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.760+0000 7f9ad77ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01a13a0 con 0x7f9ad0102690 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad5d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad0103030 0x7f9ad0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad4d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0102690 0x7f9ad00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad559b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01014a0 0x7f9ad0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad4d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0102690 0x7f9ad00739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:44982/0 (socket says 172.21.15.66:44982) 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad4d9a700 1 -- 172.21.15.66:0/193906458 learned_addr learned my addr 172.21.15.66:0/193906458 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad559b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01014a0 0x7f9ad0075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:39008/0 (socket says 172.21.15.66:39008) 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad5d9c700 1 -- 172.21.15.66:0/193906458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0102690 msgr2=0x7f9ad00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad5d9c700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0102690 0x7f9ad00739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad5d9c700 1 -- 172.21.15.66:0/193906458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01014a0 msgr2=0x7f9ad0075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad5d9c700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01014a0 0x7f9ad0075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.761+0000 7f9ad5d9c700 1 -- 172.21.15.66:0/193906458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0095f0 con 0x7f9ad0103030 2023-12-11T19:33:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.762+0000 7f9ad5d9c700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad0103030 0x7f9ad0073f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8007f10 tx=0x7f9ac800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.762+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004db0 con 0x7f9ad0103030 2023-12-11T19:33:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.762+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ac800f040 con 0x7f9ad0103030 2023-12-11T19:33:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.762+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad01a1630 con 0x7f9ad0103030 2023-12-11T19:33:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.762+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8005860 con 0x7f9ad0103030 2023-12-11T19:33:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.762+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad01a1b40 con 0x7f9ad0103030 2023-12-11T19:33:27.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.763+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad0108960 con 0x7f9ad0103030 2023-12-11T19:33:27.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.765+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9ac800dac0 con 0x7f9ad0103030 2023-12-11T19:33:27.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.766+0000 7f9ac67fc700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9abc0a1500 0x7f9abc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:27.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.767+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ac80cc520 con 0x7f9ad0103030 2023-12-11T19:33:27.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.767+0000 7f9ad559b700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9abc0a1500 0x7f9abc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:27.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.767+0000 7f9ad559b700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9abc0a1500 0x7f9abc0a39c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009990 tx=0x7f9acc003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:27.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.770+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9ac804f2f0 con 0x7f9ad0103030 2023-12-11T19:33:27.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:27 smithi066 ceph-mon[97920]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-11T19:33:27.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:27 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2499792784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:27.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:27 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/820036813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.983+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ad0068230 con 0x7f9ad0103030 2023-12-11T19:33:27.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.985+0000 7f9ac67fc700 1 -- 172.21.15.66:0/193906458 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f9ac8095b40 con 0x7f9ad0103030 2023-12-11T19:33:27.985 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:27.986 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:27.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9abc0a1500 msgr2=0x7f9abc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9abc0a1500 0x7f9abc0a39c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009990 tx=0x7f9acc003680 comp rx=0 tx=0).stop 2023-12-11T19:33:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad0103030 msgr2=0x7f9ad0073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad0103030 0x7f9ad0073f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8007f10 tx=0x7f9ac800d3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 shutdown_connections 2023-12-11T19:33:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9abc0a1500 0x7f9abc0a39c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ad01014a0 0x7f9ad0075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ad0103030 0x7f9ad0073f00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 --2- 172.21.15.66:0/193906458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ad0102690 0x7f9ad00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.987+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 >> 172.21.15.66:0/193906458 conn(0x7f9ad00fca10 msgr2=0x7f9ad0105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.988+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 shutdown_connections 2023-12-11T19:33:27.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:27.988+0000 7f9ad77ff700 1 -- 172.21.15.66:0/193906458 wait complete. 2023-12-11T19:33:28.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-11T19:31:51.764249+0000', 'modified': '2023-12-11T19:31:59.541695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24575, 'mds_1': 24547, 'mds_2': 24514, 'mds_3': 24494, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14673': {'gid': 14673, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6839/2148751938', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 2148751938}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 2148751938}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14685': {'gid': 14685, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6841/1874182059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6840', 'nonce': 1874182059}, {'type': 'v1', 'addr': '172.21.15.66:6841', 'nonce': 1874182059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/1537942709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 1537942709}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 1537942709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24467': {'gid': 24467, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.70:6833/1581467356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 1581467356}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 1581467356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.70:6835/3670150002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 3670150002}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 3670150002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.167:6835/1867143564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1867143564}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 1867143564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2775450491', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2775450491}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2775450491}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24533': {'gid': 24533, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.70:6837/280227891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 280227891}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 280227891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.167:6839/3906090427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3906090427}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3906090427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6839/725803001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6838', 'nonce': 725803001}, {'type': 'v1', 'addr': '172.21.15.70:6839', 'nonce': 725803001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-11T19:33:28.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-11T19:33:28.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-12-11T19:33:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:27 smithi070 ceph-mon[104044]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-11T19:33:28.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:27 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2499792784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:28.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:27 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/820036813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:28.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 -- 172.21.15.66:0/1594184154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b81015a0 msgr2=0x7f85b81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1594184154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b81015a0 0x7f85b81019c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f85a80099c0 tx=0x7f85a8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 -- 172.21.15.66:0/1594184154 shutdown_connections 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1594184154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b8103130 0x7f85b81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1594184154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b81015a0 0x7f85b81019c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1594184154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85b8102790 0x7f85b8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 -- 172.21.15.66:0/1594184154 >> 172.21.15.66:0/1594184154 conn(0x7f85b80fcb10 msgr2=0x7f85b80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 -- 172.21.15.66:0/1594184154 shutdown_connections 2023-12-11T19:33:28.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.216+0000 7f85be3ff700 1 -- 172.21.15.66:0/1594184154 wait complete. 2023-12-11T19:33:28.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.217+0000 7f85be3ff700 1 Processor -- start 2023-12-11T19:33:28.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.217+0000 7f85be3ff700 1 -- start start 2023-12-11T19:33:28.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85be3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85b81015a0 0x7f85b8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85be3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 0x7f85b8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85be3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b8103130 0x7f85b81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85be3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81a1180 con 0x7f85b8102790 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85be3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81a12c0 con 0x7f85b8103130 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85be3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81a1450 con 0x7f85b81015a0 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85affff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 0x7f85b8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85affff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 0x7f85b8199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58158/0 (socket says 172.21.15.66:58158) 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85affff700 1 -- 172.21.15.66:0/1691689983 learned_addr learned my addr 172.21.15.66:0/1691689983 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85bc99c700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b8103130 0x7f85b81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85b7fff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85b81015a0 0x7f85b8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.218+0000 7f85affff700 1 -- 172.21.15.66:0/1691689983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85b81015a0 msgr2=0x7f85b8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85affff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85b81015a0 0x7f85b8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85affff700 1 -- 172.21.15.66:0/1691689983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b8103130 msgr2=0x7f85b81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85affff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b8103130 0x7f85b81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85affff700 1 -- 172.21.15.66:0/1691689983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a80095f0 con 0x7f85b8102790 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85affff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 0x7f85b8199b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c9b0 tx=0x7f85a000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85b5ffb700 1 -- 172.21.15.66:0/1691689983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0007b00 con 0x7f85b8102790 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85b5ffb700 1 -- 172.21.15.66:0/1691689983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85a000ce50 con 0x7f85b8102790 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b81a1730 con 0x7f85b8102790 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85b5ffb700 1 -- 172.21.15.66:0/1691689983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0018420 con 0x7f85b8102790 2023-12-11T19:33:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.219+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b81a1c90 con 0x7f85b8102790 2023-12-11T19:33:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.222+0000 7f85b5ffb700 1 -- 172.21.15.66:0/1691689983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f85a000f450 con 0x7f85b8102790 2023-12-11T19:33:28.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 Processor -- start 2023-12-11T19:33:28.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85b5ffb700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f85a40a1500 0x7f85a40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85b5ffb700 1 -- 172.21.15.66:0/1691689983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85a00ccc20 con 0x7f85b8102790 2023-12-11T19:33:28.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 -- start start 2023-12-11T19:33:28.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85bc99c700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f85a40a1500 0x7f85a40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90dc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da9035b00 con 0x563da90da380 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da9033420 con 0x563da90dace0 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.224+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da902ed10 con 0x563da90dbbd0 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90da7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58174/0 (socket says 172.21.15.66:58174) 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 -- 172.21.15.66:0/267097201 learned_addr learned my addr 172.21.15.66:0/267097201 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85bc99c700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f85a40a1500 0x7f85a40a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f85b0003f10 tx=0x7f85b0009100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85affff700 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85bc99c700 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 -- 172.21.15.66:0/267097201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 msgr2=0x563da90dc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90dc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 -- 172.21.15.66:0/267097201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 msgr2=0x563da90db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 -- 172.21.15.66:0/267097201 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563da8eab5e0 con 0x563da90da380 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85affff700 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.225+0000 7f85b7fff700 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90da7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f85a80099c0 tx=0x7f85a8005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2201a044a97c077b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.226+0000 7f85ae7fc700 1 -- 172.21.15.66:0/267097201 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8026070 con 0x563da90da380 2023-12-11T19:33:28.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.226+0000 7f85ae7fc700 1 -- 172.21.15.66:0/267097201 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85a801e040 con 0x563da90da380 2023-12-11T19:33:28.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.226+0000 7f85ae7fc700 1 -- 172.21.15.66:0/267097201 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a801d070 con 0x563da90da380 2023-12-11T19:33:28.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.226+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/267097201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 msgr2=0x563da90da7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.226+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90da7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f85a80099c0 tx=0x7f85a8005c70 comp rx=0 tx=0).stop 2023-12-11T19:33:28.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.226+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/267097201 shutdown_connections 2023-12-11T19:33:28.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90dc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90da7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/267097201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/267097201 >> 172.21.15.66:0/267097201 conn(0x563da9009790 msgr2=0x563da90d6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/267097201 shutdown_connections 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/267097201 wait complete. 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.227+0000 7f85d1d5bb80 1 -- start start 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 Processor -- start 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90da380 0x563da90e4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da8fe8510 con 0x563da90dbbd0 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da8fe81b0 con 0x563da90dace0 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da8ead8d0 con 0x563da90da380 2023-12-11T19:33:28.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85d1d5bb80 10 client.0 _ll_register_callbacks cb 0x563da8f13a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85bc99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85bc99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90eb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58188/0 (socket says 172.21.15.66:58188) 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.228+0000 7f85bc99c700 1 -- 172.21.15.66:0/1504719162 learned_addr learned my addr 172.21.15.66:0/1504719162 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85affff700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85b7fff700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90da380 0x563da90e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85bc99c700 1 -- 172.21.15.66:0/1504719162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90da380 msgr2=0x563da90e4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85bc99c700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90da380 0x563da90e4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85bc99c700 1 -- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 msgr2=0x563da90e4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85bc99c700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85bc99c700 1 -- 172.21.15.66:0/1504719162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b81a1c90 con 0x563da90dbbd0 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85affff700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85bc99c700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90eb6d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f85b0007ab0 tx=0x7f85b0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f338d69c645767b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85977fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85977fe700 1 -- 172.21.15.66:0/1504719162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00048c0 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.229+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563da90cdaf0 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85977fe700 1 -- 172.21.15.66:0/1504719162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85b00158d0 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85977fe700 1 -- 172.21.15.66:0/1504719162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85b0015a90 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 10 client.15291 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f85b810a330 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85977fe700 1 -- 172.21.15.66:0/1504719162 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563da90cdaf0 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85977fe700 1 -- 172.21.15.66:0/1504719162 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85b0022cd0 con 0x563da90dbbd0 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 10 client.15291 resolve_mds: resolved name 'k' to daemon mds.k 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 20 client.15291 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da90efee0 0x563da90ee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 4 client.15291 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:28.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.230+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563da90a9ef0 con 0x563da90efee0 2023-12-11T19:33:28.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.231+0000 7f85b7fff700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da90efee0 0x563da90ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.232+0000 7f85977fe700 10 client.15291 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.232+0000 7f85b7fff700 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da90efee0 0x563da90ee920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a80e7af1fd43142 server_cookie=52f0355964bd940b in_seq=0 out_seq=0 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.232+0000 7f85977fe700 1 -- 172.21.15.66:0/1504719162 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563da90a9ef0 con 0x563da90efee0 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.232+0000 7f85977fe700 10 client.15291 handle_command_reply: tid=0 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 client.15291 shutdown 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 msgr2=0x563da90eb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90eb6d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f85b0007ab0 tx=0x7f85b0004b40 comp rx=0 tx=0).stop 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 shutdown_connections 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da90efee0 0x563da90ee920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90eb6d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90da380 0x563da90e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/1504719162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85977fe700 0 client.15291 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:33:28.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 >> 172.21.15.66:0/1504719162 conn(0x563da9009790 msgr2=0x563da90d59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 shutdown_connections 2023-12-11T19:33:28.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/1504719162 wait complete. 2023-12-11T19:33:28.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.233+0000 7f85d1d5bb80 20 client.15291 trim_cache size 0 max 16384 2023-12-11T19:33:28.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.252+0000 7f85d1d5bb80 1 Processor -- start 2023-12-11T19:33:28.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.252+0000 7f85d1d5bb80 1 -- start start 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90da380 0x563da90e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dace0 0x563da90eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da90dba50 con 0x563da90dbbd0 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da90cdad0 con 0x563da90da380 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da90e5050 con 0x563da90dace0 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58190/0 (socket says 172.21.15.66:58190) 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 -- 172.21.15.66:0/3564816797 learned_addr learned my addr 172.21.15.66:0/3564816797 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85b7fff700 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90da380 0x563da90e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85affff700 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dace0 0x563da90eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 -- 172.21.15.66:0/3564816797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dace0 msgr2=0x563da90eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dace0 0x563da90eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 -- 172.21.15.66:0/3564816797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90da380 msgr2=0x563da90e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90da380 0x563da90e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.253+0000 7f85bc99c700 1 -- 172.21.15.66:0/3564816797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563da8e57d90 con 0x563da90dbbd0 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.254+0000 7f85bc99c700 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90deb80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f85b00048c0 tx=0x7f85b0003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eea3a5c8bd1070 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.254+0000 7f85aeffd700 1 -- 172.21.15.66:0/3564816797 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0015920 con 0x563da90dbbd0 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.254+0000 7f85aeffd700 1 -- 172.21.15.66:0/3564816797 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85b000f040 con 0x563da90dbbd0 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.254+0000 7f85aeffd700 1 -- 172.21.15.66:0/3564816797 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0022470 con 0x563da90dbbd0 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.254+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/3564816797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 msgr2=0x563da90deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.254+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90deb80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f85b00048c0 tx=0x7f85b0003cb0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/3564816797 shutdown_connections 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90da380 0x563da90e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90dbbd0 0x563da90deb80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/3564816797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dace0 0x563da90eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/3564816797 >> 172.21.15.66:0/3564816797 conn(0x563da9009790 msgr2=0x563da90e2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/3564816797 shutdown_connections 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/3564816797 wait complete. 2023-12-11T19:33:28.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 -- start start 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.255+0000 7f85d1d5bb80 1 Processor -- start 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90eab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da90eb0b0 con 0x563da90da380 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da90eb220 con 0x563da90dace0 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563da90eb3e0 con 0x563da90dbbd0 2023-12-11T19:33:28.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85d1d5bb80 10 client.0 _ll_register_callbacks cb 0x563da8f13a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90e7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58198/0 (socket says 172.21.15.66:58198) 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85b7fff700 1 -- 172.21.15.66:0/2521605587 learned_addr learned my addr 172.21.15.66:0/2521605587 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.256+0000 7f85bc99c700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85affff700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85b7fff700 1 -- 172.21.15.66:0/2521605587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 msgr2=0x563da90eab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85b7fff700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90eab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85b7fff700 1 -- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 msgr2=0x563da90e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85b7fff700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85b7fff700 1 -- 172.21.15.66:0/2521605587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b810a330 con 0x563da90da380 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85bc99c700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:28.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85b7fff700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90e7820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f85a80099c0 tx=0x7f85a8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e09eaa441f1681c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85977fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85977fe700 1 -- 172.21.15.66:0/2521605587 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8026040 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a801e040 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.257+0000 7f85977fe700 1 -- 172.21.15.66:0/2521605587 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85a8005040 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85977fe700 1 -- 172.21.15.66:0/2521605587 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a801d040 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 10 client.15297 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563da8e40c10 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85977fe700 1 -- 172.21.15.66:0/2521605587 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85a8040020 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85977fe700 1 -- 172.21.15.66:0/2521605587 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85a800b260 con 0x563da90da380 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 10 client.15297 resolve_mds: resolved name 'k' to daemon mds.k 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 20 client.15297 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:28.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da914a670 0x563da9146af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 4 client.15297 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563da8ff4f80 con 0x563da914a670 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.258+0000 7f85affff700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da914a670 0x563da9146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85affff700 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da914a670 0x563da9146af0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd3459c77b16e007 server_cookie=a48a5ffb5f1fe585 in_seq=0 out_seq=0 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85977fe700 10 client.15297 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85977fe700 1 -- 172.21.15.66:0/2521605587 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563da8ff4f80 con 0x563da914a670 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85977fe700 10 client.15297 handle_command_reply: tid=0 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85d1d5bb80 1 client.15297 shutdown 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 msgr2=0x563da90e7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90e7820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f85a80099c0 tx=0x7f85a8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 shutdown_connections 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.259+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x563da914a670 0x563da9146af0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563da90dbbd0 0x563da90eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563da90da380 0x563da90e7820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 1 --2- 172.21.15.66:0/2521605587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563da90dace0 0x563da90e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 >> 172.21.15.66:0/2521605587 conn(0x563da9009790 msgr2=0x563da90f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85977fe700 0 client.15297 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 shutdown_connections 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 1 -- 172.21.15.66:0/2521605587 wait complete. 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85d1d5bb80 20 client.15297 trim_cache size 0 max 16384 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:28.266 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 4, 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24467, 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 89.721349910000001, 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:28.267 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 134.40693148299999 2023-12-11T19:33:28.268 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:28.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f85a40a1500 msgr2=0x7f85a40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f85a40a1500 0x7f85a40a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f85b0003f10 tx=0x7f85b0009100 comp rx=0 tx=0).stop 2023-12-11T19:33:28.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 msgr2=0x7f85b8199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.260+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 0x7f85b8199b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c9b0 tx=0x7f85a000ccc0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 shutdown_connections 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f85a40a1500 0x7f85a40a39c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b8103130 0x7f85b81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f85b81015a0 0x7f85b8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 --2- 172.21.15.66:0/1691689983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f85b8102790 0x7f85b8199b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 >> 172.21.15.66:0/1691689983 conn(0x7f85b80fcb10 msgr2=0x7f85b8107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 shutdown_connections 2023-12-11T19:33:28.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.261+0000 7f85be3ff700 1 -- 172.21.15.66:0/1691689983 wait complete. 2023-12-11T19:33:28.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:27 smithi167 ceph-mon[102750]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-11T19:33:28.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:27 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2499792784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:28.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:27 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/820036813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:28.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-12-11T19:33:28.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 -- 172.21.15.66:0/892245836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc103130 msgr2=0x7feecc1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 --2- 172.21.15.66:0/892245836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc103130 0x7feecc1099e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feec400b0d0 tx=0x7feec400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 -- 172.21.15.66:0/892245836 shutdown_connections 2023-12-11T19:33:28.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 --2- 172.21.15.66:0/892245836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feecc1015a0 0x7feecc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 --2- 172.21.15.66:0/892245836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc103130 0x7feecc1099e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 --2- 172.21.15.66:0/892245836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feecc102790 0x7feecc102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.490+0000 7feed0bff700 1 -- 172.21.15.66:0/892245836 >> 172.21.15.66:0/892245836 conn(0x7feecc0fcb10 msgr2=0x7feecc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.491+0000 7feed0bff700 1 -- 172.21.15.66:0/892245836 shutdown_connections 2023-12-11T19:33:28.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.491+0000 7feed0bff700 1 -- 172.21.15.66:0/892245836 wait complete. 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 Processor -- start 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 -- start start 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 0x7feecc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feecc102790 0x7feecc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feecc103130 0x7feecc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc1a1250 con 0x7feecc1015a0 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc1a1390 con 0x7feecc103130 2023-12-11T19:33:28.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feed0bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc1a1520 con 0x7feecc102790 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 0x7feecc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 0x7feecc199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58204/0 (socket says 172.21.15.66:58204) 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feeca59c700 1 -- 172.21.15.66:0/3166239630 learned_addr learned my addr 172.21.15.66:0/3166239630 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.492+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feecc102790 0x7feecc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feecad9d700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feecc103130 0x7feecc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feeca59c700 1 -- 172.21.15.66:0/3166239630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feecc102790 msgr2=0x7feecc199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feeca59c700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feecc102790 0x7feecc199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feeca59c700 1 -- 172.21.15.66:0/3166239630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feecc103130 msgr2=0x7feecc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feeca59c700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feecc103130 0x7feecc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feeca59c700 1 -- 172.21.15.66:0/3166239630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4009d00 con 0x7feecc1015a0 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feeca59c700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 0x7feecc199720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feebc00e970 tx=0x7feebc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feebb7fe700 1 -- 172.21.15.66:0/3166239630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc009790 con 0x7feecc1015a0 2023-12-11T19:33:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feebb7fe700 1 -- 172.21.15.66:0/3166239630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feebc00c7e0 con 0x7feecc1015a0 2023-12-11T19:33:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feecc1a1800 con 0x7feecc1015a0 2023-12-11T19:33:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feebb7fe700 1 -- 172.21.15.66:0/3166239630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc003bd0 con 0x7feecc1015a0 2023-12-11T19:33:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.493+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feecc1a1d60 con 0x7feecc1015a0 2023-12-11T19:33:28.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.496+0000 7feebb7fe700 1 -- 172.21.15.66:0/3166239630 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7feebc00c950 con 0x7feecc1015a0 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 Processor -- start 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feebb7fe700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feeb40a1500 0x7feeb40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 -- start start 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feebb7fe700 1 -- 172.21.15.66:0/3166239630 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feebc014070 con 0x7feecc1015a0 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feecad9d700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feeb40a1500 0x7feeb40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4c380 0x55c882f4c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 0x55c882f4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882ea7b00 con 0x55c882f4cce0 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882ea5420 con 0x55c882f4c380 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.498+0000 7feee45c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882ea0d10 con 0x55c882f4dbd0 2023-12-11T19:33:28.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 0x55c882f4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 0x55c882f4d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58208/0 (socket says 172.21.15.66:58208) 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 -- 172.21.15.66:0/2238000001 learned_addr learned my addr 172.21.15.66:0/2238000001 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feecad9d700 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feeca59c700 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4c380 0x55c882f4c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feecad9d700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feeb40a1500 0x7feeb40a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feec4007af0 tx=0x7feec400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 -- 172.21.15.66:0/2238000001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 msgr2=0x55c882f4e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f4e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 -- 172.21.15.66:0/2238000001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4c380 msgr2=0x55c882f4c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4c380 0x55c882f4c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 -- 172.21.15.66:0/2238000001 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c882d1d5e0 con 0x55c882f4cce0 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feeca59c700 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4c380 0x55c882f4c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.499+0000 7feec9d9b700 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 0x55c882f4d5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feec000bd70 tx=0x7feec00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2977c2e5455a3307 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feeb8ff9700 1 -- 172.21.15.66:0/2238000001 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0010ec0 con 0x55c882f4cce0 2023-12-11T19:33:28.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feeb8ff9700 1 -- 172.21.15.66:0/2238000001 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feec0015070 con 0x55c882f4cce0 2023-12-11T19:33:28.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feeb8ff9700 1 -- 172.21.15.66:0/2238000001 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0011730 con 0x55c882f4cce0 2023-12-11T19:33:28.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 -- 172.21.15.66:0/2238000001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 msgr2=0x55c882f4d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 0x55c882f4d5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feec000bd70 tx=0x7feec00106b0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 -- 172.21.15.66:0/2238000001 shutdown_connections 2023-12-11T19:33:28.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4c380 0x55c882f4c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f4e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 --2- 172.21.15.66:0/2238000001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4cce0 0x55c882f4d5f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.500+0000 7feee45c7b80 1 -- 172.21.15.66:0/2238000001 >> 172.21.15.66:0/2238000001 conn(0x55c882e7b790 msgr2=0x55c882f48d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.501+0000 7feee45c7b80 1 -- 172.21.15.66:0/2238000001 shutdown_connections 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.501+0000 7feee45c7b80 1 -- 172.21.15.66:0/2238000001 wait complete. 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.501+0000 7feee45c7b80 1 -- start start 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 Processor -- start 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f56150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f56690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882e5a510 con 0x55c882f4c380 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882e5a1b0 con 0x55c882f4dbd0 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882d1f8d0 con 0x55c882f4cce0 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feee45c7b80 10 client.0 _ll_register_callbacks cb 0x55c882d85a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:28.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.502+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f56150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f56150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58220/0 (socket says 172.21.15.66:58220) 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 -- 172.21.15.66:0/3208361737 learned_addr learned my addr 172.21.15.66:0/3208361737 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f56690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feecad9d700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 -- 172.21.15.66:0/3208361737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 msgr2=0x55c882f56690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f56690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 -- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 msgr2=0x55c882f5d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 -- 172.21.15.66:0/3208361737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec000b9a0 con 0x55c882f4c380 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feecad9d700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:28.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeca59c700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f56150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feebc096bf0 tx=0x7feebc00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f262fd4cb8abe4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3208361737 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0a2510 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3208361737 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feebc09a3e0 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.503+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c882f3faf0 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3208361737 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feebc09a5a0 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 10 client.15309 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c882cc9d90 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3208361737 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c882f3faf0 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3208361737 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feebc099670 con 0x55c882f4c380 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 10 client.15309 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 20 client.15309 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882f61da0 0x55c882f60870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 4 client.15309 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:28.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.504+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c882f1bef0 con 0x55c882f61da0 2023-12-11T19:33:28.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.505+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882f61da0 0x55c882f60870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.505+0000 7feeb1ffb700 10 client.15309 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:33:28.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.505+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882f61da0 0x55c882f60870 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=929220a1a4f27512 server_cookie=b0760b856a076749 in_seq=0 out_seq=0 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3208361737 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c882f1bef0 con 0x55c882f61da0 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feeb1ffb700 10 client.15309 handle_command_reply: tid=0 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 client.15309 shutdown 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 msgr2=0x55c882f56150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f56150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feebc096bf0 tx=0x7feebc00cd90 comp rx=0 tx=0).stop 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 shutdown_connections 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882f61da0 0x55c882f60870 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f56150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feeb1ffb700 0 client.15309 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3208361737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f56690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 >> 172.21.15.66:0/3208361737 conn(0x55c882e7b790 msgr2=0x55c882f479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 shutdown_connections 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.506+0000 7feee45c7b80 1 -- 172.21.15.66:0/3208361737 wait complete. 2023-12-11T19:33:28.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.507+0000 7feee45c7b80 20 client.15309 trim_cache size 0 max 16384 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.525+0000 7feee45c7b80 1 Processor -- start 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.525+0000 7feee45c7b80 1 -- start start 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4cce0 0x55c882f5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f50b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feee45c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882e9ce00 con 0x55c882f4c380 2023-12-11T19:33:28.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feee45c7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882e45b50 con 0x55c882f4cce0 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feee45c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882e3c360 con 0x55c882f4dbd0 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f54e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58226/0 (socket says 172.21.15.66:58226) 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 -- 172.21.15.66:0/4159327062 learned_addr learned my addr 172.21.15.66:0/4159327062 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feecad9d700 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f50b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feec9d9b700 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4cce0 0x55c882f5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 -- 172.21.15.66:0/4159327062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 msgr2=0x55c882f50b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f50b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 -- 172.21.15.66:0/4159327062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4cce0 msgr2=0x55c882f5d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4cce0 0x55c882f5d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feeca59c700 1 -- 172.21.15.66:0/4159327062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c882cb2c10 con 0x55c882f4c380 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.526+0000 7feec9d9b700 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4cce0 0x55c882f5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feeca59c700 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f54e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feebc00cc00 tx=0x7feebc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfbb98766b66336 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feeb97fa700 1 -- 172.21.15.66:0/4159327062 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0a22c0 con 0x55c882f4c380 2023-12-11T19:33:28.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feeb97fa700 1 -- 172.21.15.66:0/4159327062 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feebc0a2510 con 0x55c882f4c380 2023-12-11T19:33:28.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feeb97fa700 1 -- 172.21.15.66:0/4159327062 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc09a640 con 0x55c882f4c380 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feee45c7b80 1 -- 172.21.15.66:0/4159327062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 msgr2=0x55c882f54e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feee45c7b80 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f54e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feebc00cc00 tx=0x7feebc000bc0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feee45c7b80 1 -- 172.21.15.66:0/4159327062 shutdown_connections 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feee45c7b80 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4dbd0 0x55c882f50b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.527+0000 7feee45c7b80 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f54e30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.528+0000 7feee45c7b80 1 --2- 172.21.15.66:0/4159327062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4cce0 0x55c882f5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.528+0000 7feee45c7b80 1 -- 172.21.15.66:0/4159327062 >> 172.21.15.66:0/4159327062 conn(0x55c882e7b790 msgr2=0x55c882f54140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.528+0000 7feee45c7b80 1 -- 172.21.15.66:0/4159327062 shutdown_connections 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.528+0000 7feee45c7b80 1 -- 172.21.15.66:0/4159327062 wait complete. 2023-12-11T19:33:28.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.528+0000 7feee45c7b80 1 -- start start 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.528+0000 7feee45c7b80 1 Processor -- start 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f59820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f59d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882f5d270 con 0x55c882f4c380 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882f5d3e0 con 0x55c882f4dbd0 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c882f5d5a0 con 0x55c882f4cce0 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feee45c7b80 10 client.0 _ll_register_callbacks cb 0x55c882d85a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f59820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feeca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f59820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58238/0 (socket says 172.21.15.66:58238) 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feeca59c700 1 -- 172.21.15.66:0/3762795873 learned_addr learned my addr 172.21.15.66:0/3762795873 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feecad9d700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.529+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f59d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeca59c700 1 -- 172.21.15.66:0/3762795873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 msgr2=0x55c882f59d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeca59c700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f59d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeca59c700 1 -- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 msgr2=0x55c882f5cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeca59c700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeca59c700 1 -- 172.21.15.66:0/3762795873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c882cc9d90 con 0x55c882f4c380 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feecad9d700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeca59c700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f59820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feebc095f70 tx=0x7feebc0b81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2833815bfa43ca9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3762795873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc09a3e0 con 0x55c882f4c380 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3762795873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feebc07c020 con 0x55c882f4c380 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec0015070 con 0x55c882f4c380 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3762795873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feebc09a540 con 0x55c882f4c380 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feee45c7b80 10 client.15315 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:28.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.530+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c882cf0200 con 0x55c882f4c380 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3762795873 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feec0015070 con 0x55c882f4c380 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3762795873 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feebc099200 con 0x55c882f4c380 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feee45c7b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feee45c7b80 10 client.15315 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feee45c7b80 20 client.15315 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882fbc4b0 0x55c882fb8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feee45c7b80 4 client.15315 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c882e66f80 con 0x55c882fbc4b0 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.531+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882fbc4b0 0x55c882fb8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.532+0000 7feec9d9b700 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882fbc4b0 0x55c882fb8ac0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=107f73307cc97018 server_cookie=f870693c565c31f6 in_seq=0 out_seq=0 2023-12-11T19:33:28.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.532+0000 7feeb1ffb700 10 client.15315 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:33:28.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.532+0000 7feeb1ffb700 1 -- 172.21.15.66:0/3762795873 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c882e66f80 con 0x55c882fbc4b0 2023-12-11T19:33:28.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.532+0000 7feeb1ffb700 10 client.15315 handle_command_reply: tid=0 2023-12-11T19:33:28.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.532+0000 7feee45c7b80 1 client.15315 shutdown 2023-12-11T19:33:28.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 msgr2=0x55c882f59820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f59820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feebc095f70 tx=0x7feebc0b81e0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 shutdown_connections 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55c882fbc4b0 0x55c882fb8ac0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c882f4dbd0 0x55c882f5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c882f4c380 0x55c882f59820 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 --2- 172.21.15.66:0/3762795873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c882f4cce0 0x55c882f59d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feeb1ffb700 0 client.15315 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 >> 172.21.15.66:0/3762795873 conn(0x55c882e7b790 msgr2=0x55c882f540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 shutdown_connections 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 1 -- 172.21.15.66:0/3762795873 wait complete. 2023-12-11T19:33:28.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.533+0000 7feee45c7b80 20 client.15315 trim_cache size 0 max 16384 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 3, 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24494, 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 91.017822140000007, 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:28.538 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 125.14877632 2023-12-11T19:33:28.539 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:28.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feeb40a1500 msgr2=0x7feeb40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feeb40a1500 0x7feeb40a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feec4007af0 tx=0x7feec400b040 comp rx=0 tx=0).stop 2023-12-11T19:33:28.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 msgr2=0x7feecc199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 0x7feecc199720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feebc00e970 tx=0x7feebc00ec80 comp rx=0 tx=0).stop 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 shutdown_connections 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feeb40a1500 0x7feeb40a39c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feecc103130 0x7feecc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feecc1015a0 0x7feecc199720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 --2- 172.21.15.66:0/3166239630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feecc102790 0x7feecc199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 >> 172.21.15.66:0/3166239630 conn(0x7feecc0fcb10 msgr2=0x7feecc107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 shutdown_connections 2023-12-11T19:33:28.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.534+0000 7feed0bff700 1 -- 172.21.15.66:0/3166239630 wait complete. 2023-12-11T19:33:28.553 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-12-11T19:33:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/2905576378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 msgr2=0x7f4d581019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/2905576378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 0x7f4d581019c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d480099c0 tx=0x7f4d48009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/2905576378 shutdown_connections 2023-12-11T19:33:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/2905576378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 0x7f4d581019c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/2905576378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58103130 0x7f4d581099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/2905576378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d58102790 0x7f4d58102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/2905576378 >> 172.21.15.66:0/2905576378 conn(0x7f4d580fcb10 msgr2=0x7f4d580fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/2905576378 shutdown_connections 2023-12-11T19:33:28.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.762+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/2905576378 wait complete. 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.763+0000 7f4d5c9ff700 1 Processor -- start 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.763+0000 7f4d5c9ff700 1 -- start start 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5c9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 0x7f4d58199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5c9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 0x7f4d58199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5c9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d58103130 0x7f4d581a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5c9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a1230 con 0x7f4d58102790 2023-12-11T19:33:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5c9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a1370 con 0x7f4d581015a0 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5c9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a1500 con 0x7f4d58103130 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 0x7f4d58199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 0x7f4d58199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58258/0 (socket says 172.21.15.66:58258) 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4186109448 learned_addr learned my addr 172.21.15.66:0/4186109448 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.764+0000 7f4d5659c700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 0x7f4d58199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4186109448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d58103130 msgr2=0x7f4d581a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d58103130 0x7f4d581a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4186109448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 msgr2=0x7f4d58199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 0x7f4d58199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4186109448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50009c40 con 0x7f4d58102790 2023-12-11T19:33:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 0x7f4d58199c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00d950 tx=0x7f4d4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d477fe700 1 -- 172.21.15.66:0/4186109448 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c009930 con 0x7f4d58102790 2023-12-11T19:33:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d477fe700 1 -- 172.21.15.66:0/4186109448 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d4c010460 con 0x7f4d58102790 2023-12-11T19:33:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d480095f0 con 0x7f4d58102790 2023-12-11T19:33:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d477fe700 1 -- 172.21.15.66:0/4186109448 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00f620 con 0x7f4d58102790 2023-12-11T19:33:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.765+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a1b60 con 0x7f4d58102790 2023-12-11T19:33:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.768+0000 7f4d477fe700 1 -- 172.21.15.66:0/4186109448 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4d4c0105d0 con 0x7f4d58102790 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 Processor -- start 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 -- start start 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d477fe700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4d400a1480 0x7f4d400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d477fe700 1 -- 172.21.15.66:0/4186109448 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4d4c0ccb10 con 0x7f4d58102790 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc4d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc4f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d56d9d700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4d400a1480 0x7f4d400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cba8b00 con 0x56499cc4dce0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cba6420 con 0x56499cc4d380 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d70416b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cba1d10 con 0x56499cc4ebd0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.770+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc4e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58270/0 (socket says 172.21.15.66:58270) 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4183084274 learned_addr learned my addr 172.21.15.66:0/4183084274 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d56d9d700 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc4f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4183084274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 msgr2=0x56499cc4f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc4f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4183084274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 msgr2=0x56499cc4d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d5659c700 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc4d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc4d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 -- 172.21.15.66:0/4183084274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56499ca1e5e0 con 0x56499cc4dce0 2023-12-11T19:33:28.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d56d9d700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4d400a1480 0x7f4d400a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c820 tx=0x7f4d5000cbe0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc4e5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c099af0 tx=0x7f4d4c099750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e54eb5a3914da0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d44ff9700 1 -- 172.21.15.66:0/4183084274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c099df0 con 0x56499cc4dce0 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d44ff9700 1 -- 172.21.15.66:0/4183084274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d4c09df40 con 0x56499cc4dce0 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.771+0000 7f4d44ff9700 1 -- 172.21.15.66:0/4183084274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0a1cf0 con 0x56499cc4dce0 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 -- 172.21.15.66:0/4183084274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 msgr2=0x56499cc4e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc4e5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c099af0 tx=0x7f4d4c099750 comp rx=0 tx=0).stop 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 -- 172.21.15.66:0/4183084274 shutdown_connections 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc4d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc4f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 --2- 172.21.15.66:0/4183084274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc4e5f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 -- 172.21.15.66:0/4183084274 >> 172.21.15.66:0/4183084274 conn(0x56499cb7c790 msgr2=0x56499cc49d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 -- 172.21.15.66:0/4183084274 shutdown_connections 2023-12-11T19:33:28.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.772+0000 7f4d70416b80 1 -- 172.21.15.66:0/4183084274 wait complete. 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 -- start start 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 Processor -- start 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cb5b510 con 0x56499cc4dce0 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cb5b1b0 con 0x56499cc4d380 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.773+0000 7f4d70416b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499ca208d0 con 0x56499cc4ebd0 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d70416b80 10 client.0 _ll_register_callbacks cb 0x56499ca86a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc57730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58282/0 (socket says 172.21.15.66:58282) 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 -- 172.21.15.66:0/2771685903 learned_addr learned my addr 172.21.15.66:0/2771685903 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d56d9d700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d5659c700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 -- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 msgr2=0x56499cc5e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 -- 172.21.15.66:0/2771685903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 msgr2=0x56499cc571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 -- 172.21.15.66:0/2771685903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a1b60 con 0x56499cc4dce0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d5659c700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc57730 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c010880 tx=0x7f4d4c098190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee55cf4fcde7a3bc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.774+0000 7f4d3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d3dffb700 1 -- 172.21.15.66:0/2771685903 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0addd0 con 0x56499cc4dce0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d3dffb700 1 -- 172.21.15.66:0/2771685903 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4d4c099df0 con 0x56499cc4dce0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56499cc40af0 con 0x56499cc4dce0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d3dffb700 1 -- 172.21.15.66:0/2771685903 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d4c099560 con 0x56499cc4dce0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 10 client.15327 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56499c9cad90 con 0x56499cc4dce0 2023-12-11T19:33:28.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d3dffb700 1 -- 172.21.15.66:0/2771685903 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d4c07d020 con 0x56499cc4dce0 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d3dffb700 1 -- 172.21.15.66:0/2771685903 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d4c05fce0 con 0x56499cc4dce0 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 10 client.15327 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 20 client.15327 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499cc629b0 0x56499cc61740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 4 client.15327 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.775+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56499cc1cef0 con 0x56499cc629b0 2023-12-11T19:33:28.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.776+0000 7f4d5659c700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499cc629b0 0x56499cc61740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.776+0000 7f4d5659c700 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499cc629b0 0x56499cc61740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f6ed5b36d665aef server_cookie=d86b8cc49822acc8 in_seq=0 out_seq=0 2023-12-11T19:33:28.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.776+0000 7f4d3dffb700 10 client.15327 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:28.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.777+0000 7f4d3dffb700 1 -- 172.21.15.66:0/2771685903 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56499cc1cef0 con 0x56499cc629b0 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.777+0000 7f4d3dffb700 10 client.15327 handle_command_reply: tid=0 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 client.15327 shutdown 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 msgr2=0x56499cc57730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc57730 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c010880 tx=0x7f4d4c098190 comp rx=0 tx=0).stop 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 shutdown_connections 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499cc629b0 0x56499cc61740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d3dffb700 0 client.15327 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:28.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 --2- 172.21.15.66:0/2771685903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc57730 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 >> 172.21.15.66:0/2771685903 conn(0x56499cb7c790 msgr2=0x56499cc489f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 shutdown_connections 2023-12-11T19:33:28.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 1 -- 172.21.15.66:0/2771685903 wait complete. 2023-12-11T19:33:28.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.778+0000 7f4d70416b80 20 client.15327 trim_cache size 0 max 16384 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.797+0000 7f4d70416b80 1 Processor -- start 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 -- start start 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4d380 0x56499cc55170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4ebd0 0x56499cc61740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cb46b90 con 0x56499cc4dce0 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cb3d360 con 0x56499cc4ebd0 2023-12-11T19:33:28.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d70416b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cc44740 con 0x56499cc4d380 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58286/0 (socket says 172.21.15.66:58286) 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d55d9b700 1 -- 172.21.15.66:0/1487095378 learned_addr learned my addr 172.21.15.66:0/1487095378 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d56d9d700 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4ebd0 0x56499cc61740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.798+0000 7f4d5659c700 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4d380 0x56499cc55170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.799+0000 7f4d55d9b700 1 -- 172.21.15.66:0/1487095378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4d380 msgr2=0x56499cc55170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.799+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4d380 0x56499cc55170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.799+0000 7f4d55d9b700 1 -- 172.21.15.66:0/1487095378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4ebd0 msgr2=0x56499cc61740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.799+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4ebd0 0x56499cc61740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.799+0000 7f4d55d9b700 1 -- 172.21.15.66:0/1487095378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56499c9b3c10 con 0x56499cc4dce0 2023-12-11T19:33:28.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.799+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5e690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00b2f0 tx=0x7f4d4c098200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc95289d56f2e35 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.800+0000 7f4d457fa700 1 -- 172.21.15.66:0/1487095378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c099560 con 0x56499cc4dce0 2023-12-11T19:33:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.800+0000 7f4d457fa700 1 -- 172.21.15.66:0/1487095378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d4c07d070 con 0x56499cc4dce0 2023-12-11T19:33:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.800+0000 7f4d457fa700 1 -- 172.21.15.66:0/1487095378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c099e80 con 0x56499cc4dce0 2023-12-11T19:33:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 -- 172.21.15.66:0/1487095378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 msgr2=0x56499cc5e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5e690 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00b2f0 tx=0x7f4d4c098200 comp rx=0 tx=0).stop 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 -- 172.21.15.66:0/1487095378 shutdown_connections 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4ebd0 0x56499cc61740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4d380 0x56499cc55170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 --2- 172.21.15.66:0/1487095378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5e690 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 -- 172.21.15.66:0/1487095378 >> 172.21.15.66:0/1487095378 conn(0x56499cb7c790 msgr2=0x56499cc56700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 -- 172.21.15.66:0/1487095378 shutdown_connections 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 -- 172.21.15.66:0/1487095378 wait complete. 2023-12-11T19:33:28.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.801+0000 7f4d70416b80 1 -- start start 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 Processor -- start 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cc5e4d0 con 0x56499cc4dce0 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cc5e640 con 0x56499cc4d380 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56499cc5e800 con 0x56499cc4ebd0 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d70416b80 10 client.0 _ll_register_callbacks cb 0x56499ca86a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d55d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58292/0 (socket says 172.21.15.66:58292) 2023-12-11T19:33:28.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d55d9b700 1 -- 172.21.15.66:0/3114727713 learned_addr learned my addr 172.21.15.66:0/3114727713 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d5659c700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.802+0000 7f4d56d9d700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d55d9b700 1 -- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 msgr2=0x56499cc5df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d55d9b700 1 -- 172.21.15.66:0/3114727713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 msgr2=0x56499cc5aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc5aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d55d9b700 1 -- 172.21.15.66:0/3114727713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56499c9cad90 con 0x56499cc4dce0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d55d9b700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5b010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c099af0 tx=0x7f4d4c098190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334b73c2700cb5a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d3dffb700 1 -- 172.21.15.66:0/3114727713 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c099560 con 0x56499cc4dce0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d4c09df40 con 0x56499cc4dce0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d3dffb700 1 -- 172.21.15.66:0/3114727713 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4d4c07e080 con 0x56499cc4dce0 2023-12-11T19:33:28.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.803+0000 7f4d3dffb700 1 -- 172.21.15.66:0/3114727713 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d4c099e60 con 0x56499cc4dce0 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 10 client.15333 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56499c9f1200 con 0x56499cc4dce0 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d3dffb700 1 -- 172.21.15.66:0/3114727713 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d4c09df40 con 0x56499cc4dce0 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d3dffb700 1 -- 172.21.15.66:0/3114727713 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d4c009b90 con 0x56499cc4dce0 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 10 client.15333 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 20 client.15333 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499ccbd100 0x56499ccb9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 4 client.15333 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56499cb67f80 con 0x56499ccbd100 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.804+0000 7f4d5659c700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499ccbd100 0x56499ccb9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.805+0000 7f4d3dffb700 10 client.15333 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.805+0000 7f4d5659c700 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499ccbd100 0x56499ccb9780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4eb285c51a0fd144 server_cookie=d5370680ca40562 in_seq=0 out_seq=0 2023-12-11T19:33:28.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.805+0000 7f4d3dffb700 1 -- 172.21.15.66:0/3114727713 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56499cb67f80 con 0x56499ccbd100 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.805+0000 7f4d3dffb700 10 client.15333 handle_command_reply: tid=0 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.805+0000 7f4d70416b80 1 client.15333 shutdown 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 msgr2=0x56499cc5b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5b010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c099af0 tx=0x7f4d4c098190 comp rx=0 tx=0).stop 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 shutdown_connections 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56499ccbd100 0x56499ccb9780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56499cc4d380 0x56499cc5aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d3dffb700 0 client.15333 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56499cc4ebd0 0x56499cc5df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 --2- 172.21.15.66:0/3114727713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56499cc4dce0 0x56499cc5b010 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 >> 172.21.15.66:0/3114727713 conn(0x56499cb7c790 msgr2=0x56499cc6b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 shutdown_connections 2023-12-11T19:33:28.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 1 -- 172.21.15.66:0/3114727713 wait complete. 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.806+0000 7f4d70416b80 20 client.15333 trim_cache size 0 max 16384 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24514, 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 92.322063205999996, 2023-12-11T19:33:28.811 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 122.170410218 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4d400a1480 msgr2=0x7f4d400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4d400a1480 0x7f4d400a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c820 tx=0x7f4d5000cbe0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 msgr2=0x7f4d58199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 0x7f4d58199c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00d950 tx=0x7f4d4c00dc60 comp rx=0 tx=0).stop 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 shutdown_connections 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4d400a1480 0x7f4d400a3940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d581015a0 0x7f4d58199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d58103130 0x7f4d581a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 --2- 172.21.15.66:0/4186109448 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d58102790 0x7f4d58199c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 >> 172.21.15.66:0/4186109448 conn(0x7f4d580fcb10 msgr2=0x7f4d58108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 shutdown_connections 2023-12-11T19:33:28.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:28.807+0000 7f4d5c9ff700 1 -- 172.21.15.66:0/4186109448 wait complete. 2023-12-11T19:33:28.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:28 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/193906458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:28.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-11T19:33:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 -- 172.21.15.66:0/97954354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 msgr2=0x7f3b801019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 --2- 172.21.15.66:0/97954354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b801019c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0099c0 tx=0x7f3b7c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 -- 172.21.15.66:0/97954354 shutdown_connections 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 --2- 172.21.15.66:0/97954354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80103130 0x7f3b801099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 --2- 172.21.15.66:0/97954354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b801019c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 --2- 172.21.15.66:0/97954354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b80102790 0x7f3b80102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 -- 172.21.15.66:0/97954354 >> 172.21.15.66:0/97954354 conn(0x7f3b800fcb10 msgr2=0x7f3b800fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 -- 172.21.15.66:0/97954354 shutdown_connections 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.037+0000 7f3b877ff700 1 -- 172.21.15.66:0/97954354 wait complete. 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.038+0000 7f3b877ff700 1 Processor -- start 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.038+0000 7f3b877ff700 1 -- start start 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.038+0000 7f3b877ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b80199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.038+0000 7f3b877ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80102790 0x7f3b80199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b877ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b80103130 0x7f3b801a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b877ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b801a1230 con 0x7f3b801015a0 2023-12-11T19:33:29.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b877ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b801a1370 con 0x7f3b80103130 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b877ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b801a1500 con 0x7f3b80102790 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b80199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b80199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58310/0 (socket says 172.21.15.66:58310) 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 -- 172.21.15.66:0/1462316997 learned_addr learned my addr 172.21.15.66:0/1462316997 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80102790 0x7f3b80199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b80103130 0x7f3b801a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 -- 172.21.15.66:0/1462316997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80102790 msgr2=0x7f3b80199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80102790 0x7f3b80199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 -- 172.21.15.66:0/1462316997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b80103130 msgr2=0x7f3b801a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b80103130 0x7f3b801a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 -- 172.21.15.66:0/1462316997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0095f0 con 0x7f3b801015a0 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80102790 0x7f3b80199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.039+0000 7f3b8559b700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b80199700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009f50 tx=0x7f3b7c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.040+0000 7f3b767fc700 1 -- 172.21.15.66:0/1462316997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c01d070 con 0x7f3b801015a0 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.040+0000 7f3b767fc700 1 -- 172.21.15.66:0/1462316997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b7c00bda0 con 0x7f3b801015a0 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.040+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b801a1780 con 0x7f3b801015a0 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.040+0000 7f3b767fc700 1 -- 172.21.15.66:0/1462316997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c022dc0 con 0x7f3b801015a0 2023-12-11T19:33:29.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.040+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b801a1c00 con 0x7f3b801015a0 2023-12-11T19:33:29.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.043+0000 7f3b767fc700 1 -- 172.21.15.66:0/1462316997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3b7c004000 con 0x7f3b801015a0 2023-12-11T19:33:29.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.044+0000 7f3b9b102b80 1 Processor -- start 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.044+0000 7f3b767fc700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b6c0a1500 0x7f3b6c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.044+0000 7f3b9b102b80 1 -- start start 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.044+0000 7f3b767fc700 1 -- 172.21.15.66:0/1462316997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b7c0ce140 con 0x7f3b801015a0 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.044+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 0x5621718537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b6c0a1500 0x7f3b6c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x5621718545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x562171855050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621717aeb00 con 0x562171853380 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621717ac420 con 0x562171854bd0 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621717a7d10 con 0x562171853ce0 2023-12-11T19:33:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 0x5621718537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 0x5621718537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58326/0 (socket says 172.21.15.66:58326) 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 -- 172.21.15.66:0/585020381 learned_addr learned my addr 172.21.15.66:0/585020381 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 -- 172.21.15.66:0/585020381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 msgr2=0x5621718545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x5621718545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 -- 172.21.15.66:0/585020381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 msgr2=0x562171855050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x562171855050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 -- 172.21.15.66:0/585020381 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621716245e0 con 0x562171853380 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b8559b700 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 0x5621718537a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0979b0 tx=0x7f3b7c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a5a4be312f9132 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.045+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b6c0a1500 0x7f3b6c0a39c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009550 tx=0x7f3b780077e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b6b7fe700 1 -- 172.21.15.66:0/585020381 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c09b4b0 con 0x562171853380 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b6b7fe700 1 -- 172.21.15.66:0/585020381 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b7c09f600 con 0x562171853380 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b6b7fe700 1 -- 172.21.15.66:0/585020381 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c09a9a0 con 0x562171853380 2023-12-11T19:33:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 -- 172.21.15.66:0/585020381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 msgr2=0x5621718537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 0x5621718537a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0979b0 tx=0x7f3b7c00bf90 comp rx=0 tx=0).stop 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 -- 172.21.15.66:0/585020381 shutdown_connections 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x562171855050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853380 0x5621718537a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/585020381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x5621718545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.046+0000 7f3b9b102b80 1 -- 172.21.15.66:0/585020381 >> 172.21.15.66:0/585020381 conn(0x562171782790 msgr2=0x56217184fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.047+0000 7f3b9b102b80 1 -- 172.21.15.66:0/585020381 shutdown_connections 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.047+0000 7f3b9b102b80 1 -- 172.21.15.66:0/585020381 wait complete. 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.047+0000 7f3b9b102b80 1 -- start start 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.047+0000 7f3b9b102b80 1 Processor -- start 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217185ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x56217185d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x5621718643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562171761510 con 0x562171853ce0 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621717611b0 con 0x562171854bd0 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621716268d0 con 0x562171853380 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b9b102b80 10 client.0 _ll_register_callbacks cb 0x56217168ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x56217185d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x56217185d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58330/0 (socket says 172.21.15.66:58330) 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 -- 172.21.15.66:0/3550428077 learned_addr learned my addr 172.21.15.66:0/3550428077 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b8559b700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217185ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x5621718643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 -- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 msgr2=0x56217185ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217185ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 -- 172.21.15.66:0/3550428077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 msgr2=0x5621718643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.048+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x5621718643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b84d9a700 1 -- 172.21.15.66:0/3550428077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b801a1c00 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x56217185d370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3b70003c10 tx=0x7f3b70003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f2e8f26d6f2fb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3550428077 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000fdd0 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3550428077 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b7001a460 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562171846af0 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3550428077 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b7001a620 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b9b102b80 10 client.15345 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.049+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5621715d0d90 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3550428077 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562171846af0 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3550428077 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b70011810 con 0x562171853ce0 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b9b102b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b9b102b80 10 client.15345 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T19:33:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b9b102b80 20 client.15345 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x562171868ae0 0x5621718675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b9b102b80 4 client.15345 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562171822ef0 con 0x562171868ae0 2023-12-11T19:33:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.050+0000 7f3b8559b700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x562171868ae0 0x5621718675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.051+0000 7f3b8559b700 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x562171868ae0 0x5621718675b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=337c5b4b04aa7cca server_cookie=cba02740506479db in_seq=0 out_seq=0 2023-12-11T19:33:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.051+0000 7f3b68ff9700 10 client.15345 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3550428077 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562171822ef0 con 0x562171868ae0 2023-12-11T19:33:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b68ff9700 10 client.15345 handle_command_reply: tid=0 2023-12-11T19:33:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 client.15345 shutdown 2023-12-11T19:33:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 msgr2=0x56217185d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x56217185d370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3b70003c10 tx=0x7f3b70003f20 comp rx=0 tx=0).stop 2023-12-11T19:33:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 shutdown_connections 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x562171868ae0 0x5621718675b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x5621718643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217185ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b68ff9700 0 client.15345 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3550428077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x56217185d370 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 >> 172.21.15.66:0/3550428077 conn(0x562171782790 msgr2=0x56217184e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 shutdown_connections 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.052+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3550428077 wait complete. 2023-12-11T19:33:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.053+0000 7f3b9b102b80 20 client.15345 trim_cache size 0 max 16384 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.071+0000 7f3b9b102b80 1 Processor -- start 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.071+0000 7f3b9b102b80 1 -- start start 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217184f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x562171864370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x56217184fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621717a3e00 con 0x562171853ce0 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56217174cb50 con 0x562171854bd0 2023-12-11T19:33:29.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562171743360 con 0x562171853380 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x562171864370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x562171864370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58340/0 (socket says 172.21.15.66:58340) 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 -- 172.21.15.66:0/1040498273 learned_addr learned my addr 172.21.15.66:0/1040498273 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b8559b700 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217184f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x56217184fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 -- 172.21.15.66:0/1040498273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 msgr2=0x56217184f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217184f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 -- 172.21.15.66:0/1040498273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 msgr2=0x56217184fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x56217184fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b84d9a700 1 -- 172.21.15.66:0/1040498273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621715b9c10 con 0x562171853ce0 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.072+0000 7f3b8559b700 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217184f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.073+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x562171864370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b70001eb0 tx=0x7f3b70004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a932dbf36f43e9a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.073+0000 7f3b6bfff700 1 -- 172.21.15.66:0/1040498273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000fbd0 con 0x562171853ce0 2023-12-11T19:33:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.073+0000 7f3b6bfff700 1 -- 172.21.15.66:0/1040498273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b70020030 con 0x562171853ce0 2023-12-11T19:33:29.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.073+0000 7f3b6bfff700 1 -- 172.21.15.66:0/1040498273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70005620 con 0x562171853ce0 2023-12-11T19:33:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.073+0000 7f3b9b102b80 1 -- 172.21.15.66:0/1040498273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 msgr2=0x562171864370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.073+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x562171864370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b70001eb0 tx=0x7f3b70004070 comp rx=0 tx=0).stop 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 -- 172.21.15.66:0/1040498273 shutdown_connections 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171854bd0 0x56217184fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853380 0x56217184f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/1040498273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171853ce0 0x562171864370 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 -- 172.21.15.66:0/1040498273 >> 172.21.15.66:0/1040498273 conn(0x562171782790 msgr2=0x56217185c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 -- 172.21.15.66:0/1040498273 shutdown_connections 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 -- 172.21.15.66:0/1040498273 wait complete. 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 -- start start 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.074+0000 7f3b9b102b80 1 Processor -- start 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171853380 0x56217185eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x56217185f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 0x562171863e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621718643a0 con 0x562171854bd0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562171864510 con 0x562171853380 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562171864680 con 0x562171853ce0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b9b102b80 10 client.0 _ll_register_callbacks cb 0x56217168ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 0x562171863e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 0x562171863e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58352/0 (socket says 172.21.15.66:58352) 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 -- 172.21.15.66:0/3225267257 learned_addr learned my addr 172.21.15.66:0/3225267257 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b84d9a700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x56217185f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b8559b700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171853380 0x56217185eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 -- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 msgr2=0x56217185f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x56217185f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 -- 172.21.15.66:0/3225267257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171853380 msgr2=0x56217185eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171853380 0x56217185eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.075+0000 7f3b85d9c700 1 -- 172.21.15.66:0/3225267257 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621715d0d90 con 0x562171854bd0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b85d9c700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 0x562171863e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b78016a60 tx=0x7f3b78004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85174bb364b1f14 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3225267257 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7801f040 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b7c09f600 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3225267257 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b78019ab0 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3225267257 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b78019c70 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b9b102b80 10 client.15351 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.076+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5621715f7200 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3225267257 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b7c09f600 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3225267257 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b7802f3e0 con 0x562171854bd0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b9b102b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b9b102b80 10 client.15351 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b9b102b80 20 client.15351 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5621718c3df0 0x5621718c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b9b102b80 4 client.15351 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56217176df80 con 0x5621718c3df0 2023-12-11T19:33:29.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.077+0000 7f3b8559b700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5621718c3df0 0x5621718c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b8559b700 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5621718c3df0 0x5621718c0230 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aea43bc786131e7f server_cookie=e817701d1528ff96 in_seq=0 out_seq=0 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b68ff9700 10 client.15351 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b68ff9700 1 -- 172.21.15.66:0/3225267257 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56217176df80 con 0x5621718c3df0 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b68ff9700 10 client.15351 handle_command_reply: tid=0 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 client.15351 shutdown 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 msgr2=0x562171863e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 0x562171863e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b78016a60 tx=0x7f3b78004b40 comp rx=0 tx=0).stop 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 shutdown_connections 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5621718c3df0 0x5621718c0230 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562171853380 0x56217185eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562171854bd0 0x562171863e60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 --2- 172.21.15.66:0/3225267257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562171853ce0 0x56217185f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b68ff9700 0 client.15351 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 >> 172.21.15.66:0/3225267257 conn(0x562171782790 msgr2=0x56217184f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 shutdown_connections 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.078+0000 7f3b9b102b80 1 -- 172.21.15.66:0/3225267257 wait complete. 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b9b102b80 20 client.15351 trim_cache size 0 max 16384 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24547, 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:29.083 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 93.579178772000006, 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 112.93803531 2023-12-11T19:33:29.084 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b6c0a1500 msgr2=0x7f3b6c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b6c0a1500 0x7f3b6c0a39c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009550 tx=0x7f3b780077e0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 msgr2=0x7f3b80199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b80199700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009f50 tx=0x7f3b7c003b00 comp rx=0 tx=0).stop 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 shutdown_connections 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b6c0a1500 0x7f3b6c0a39c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b80103130 0x7f3b801a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b801015a0 0x7f3b80199700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 --2- 172.21.15.66:0/1462316997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b80102790 0x7f3b80199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 >> 172.21.15.66:0/1462316997 conn(0x7f3b800fcb10 msgr2=0x7f3b80108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 shutdown_connections 2023-12-11T19:33:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.079+0000 7f3b877ff700 1 -- 172.21.15.66:0/1462316997 wait complete. 2023-12-11T19:33:29.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-11T19:33:29.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:28 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/193906458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:29.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:28 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/193906458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 -- 172.21.15.66:0/3653099137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c103130 msgr2=0x7f4b4c1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3653099137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c103130 0x7f4b4c1099e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400b0d0 tx=0x7f4b4400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 -- 172.21.15.66:0/3653099137 shutdown_connections 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3653099137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c103130 0x7f4b4c1099e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3653099137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c1015a0 0x7f4b4c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3653099137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c102790 0x7f4b4c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 -- 172.21.15.66:0/3653099137 >> 172.21.15.66:0/3653099137 conn(0x7f4b4c0fcb10 msgr2=0x7f4b4c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 -- 172.21.15.66:0/3653099137 shutdown_connections 2023-12-11T19:33:29.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.306+0000 7f4b545ff700 1 -- 172.21.15.66:0/3653099137 wait complete. 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 Processor -- start 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 -- start start 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c1015a0 0x7f4b4c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 0x7f4b4c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c103130 0x7f4b4c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c19efe0 con 0x7f4b4c102790 2023-12-11T19:33:29.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c19f120 con 0x7f4b4c1015a0 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.307+0000 7f4b545ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c19f2b0 con 0x7f4b4c103130 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 0x7f4b4c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 0x7f4b4c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58360/0 (socket says 172.21.15.66:58360) 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 -- 172.21.15.66:0/3961695699 learned_addr learned my addr 172.21.15.66:0/3961695699 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c103130 0x7f4b4c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b5239b700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c1015a0 0x7f4b4c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 -- 172.21.15.66:0/3961695699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c103130 msgr2=0x7f4b4c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c103130 0x7f4b4c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 -- 172.21.15.66:0/3961695699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c1015a0 msgr2=0x7f4b4c1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c1015a0 0x7f4b4c1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 -- 172.21.15.66:0/3961695699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44009d00 con 0x7f4b4c102790 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c103130 0x7f4b4c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:29.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b5239b700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c1015a0 0x7f4b4c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.308+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 0x7f4b4c1979f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c9b0 tx=0x7f4b3c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.309+0000 7f4b437fe700 1 -- 172.21.15.66:0/3961695699 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c007b00 con 0x7f4b4c102790 2023-12-11T19:33:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.309+0000 7f4b437fe700 1 -- 172.21.15.66:0/3961695699 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b3c007c60 con 0x7f4b4c102790 2023-12-11T19:33:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.309+0000 7f4b437fe700 1 -- 172.21.15.66:0/3961695699 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c018660 con 0x7f4b4c102790 2023-12-11T19:33:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.309+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c19f590 con 0x7f4b4c102790 2023-12-11T19:33:29.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.309+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c19faa0 con 0x7f4b4c102790 2023-12-11T19:33:29.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.312+0000 7f4b437fe700 1 -- 172.21.15.66:0/3961695699 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4b3c00f450 con 0x7f4b4c102790 2023-12-11T19:33:29.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.313+0000 7f4b67fc7b80 1 Processor -- start 2023-12-11T19:33:29.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.313+0000 7f4b437fe700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4b380a1500 0x7f4b380a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.313+0000 7f4b67fc7b80 1 -- start start 2023-12-11T19:33:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b437fe700 1 -- 172.21.15.66:0/3961695699 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b3c0cccd0 con 0x7f4b4c102790 2023-12-11T19:33:29.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4b380a1500 0x7f4b380a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edad0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad10f0 0x55d2edad1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edad2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2eda27b00 con 0x55d2edad10f0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2eda25420 con 0x55d2edad1fe0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2eda20d10 con 0x55d2edad0790 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b51b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad10f0 0x55d2edad1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b5239b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edad0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b5239b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edad0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:45186/0 (socket says 172.21.15.66:45186) 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b5239b700 1 -- 172.21.15.66:0/2659477120 learned_addr learned my addr 172.21.15.66:0/2659477120 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.314+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edad2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4b380a1500 0x7f4b380a39c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4b44007f80 tx=0x7f4b440074e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b51b9a700 1 -- 172.21.15.66:0/2659477120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 msgr2=0x55d2edad0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edad0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b51b9a700 1 -- 172.21.15.66:0/2659477120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 msgr2=0x55d2edad2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edad2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b51b9a700 1 -- 172.21.15.66:0/2659477120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ed89d5e0 con 0x55d2edad10f0 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edad2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad10f0 0x55d2edad1a00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c099cb0 tx=0x7f4b3c099a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a99c4241fd21f76 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b40bfe700 1 -- 172.21.15.66:0/2659477120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c09a6b0 con 0x55d2edad10f0 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b40bfe700 1 -- 172.21.15.66:0/2659477120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b3c099fb0 con 0x55d2edad10f0 2023-12-11T19:33:29.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.315+0000 7f4b40bfe700 1 -- 172.21.15.66:0/2659477120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00f700 con 0x55d2edad10f0 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/2659477120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad10f0 msgr2=0x55d2edad1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad10f0 0x55d2edad1a00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c099cb0 tx=0x7f4b3c099a00 comp rx=0 tx=0).stop 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/2659477120 shutdown_connections 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edad2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edad0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/2659477120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad10f0 0x55d2edad1a00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/2659477120 >> 172.21.15.66:0/2659477120 conn(0x55d2ed9fb790 msgr2=0x55d2edac8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/2659477120 shutdown_connections 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.316+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/2659477120 wait complete. 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 -- start start 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 Processor -- start 2023-12-11T19:33:29.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 0x55d2edada5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edadaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edae1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ed9da510 con 0x55d2edad0790 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ed9da1b0 con 0x55d2edad1fe0 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.317+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ed89f8d0 con 0x55d2edad10f0 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b67fc7b80 10 client.0 _ll_register_callbacks cb 0x55d2ed905a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 0x55d2edada5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 0x55d2edada5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58392/0 (socket says 172.21.15.66:58392) 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 -- 172.21.15.66:0/1752906091 learned_addr learned my addr 172.21.15.66:0/1752906091 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edadaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edae1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 -- 172.21.15.66:0/1752906091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 msgr2=0x55d2edadaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edadaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 -- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 msgr2=0x55d2edae1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edae1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.318+0000 7f4b5239b700 1 -- 172.21.15.66:0/1752906091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c19faa0 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b5239b700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 0x55d2edada5a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4b48012740 tx=0x7f4b48012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd1990f2beb464c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b35ffb700 1 -- 172.21.15.66:0/1752906091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48017070 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b35ffb700 1 -- 172.21.15.66:0/1752906091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b48011710 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2edabfaf0 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b35ffb700 1 -- 172.21.15.66:0/1752906091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b480118d0 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b67fc7b80 10 client.15363 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d2ed849d90 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b35ffb700 1 -- 172.21.15.66:0/1752906091 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d2edabfaf0 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.319+0000 7f4b35ffb700 1 -- 172.21.15.66:0/1752906091 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b4800fb70 con 0x55d2edad0790 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b67fc7b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:29.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b67fc7b80 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b67fc7b80 20 client.15363 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edae6270 0x55d2edae4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b67fc7b80 4 client.15363 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2eda9bef0 con 0x55d2edae6270 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.320+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edae6270 0x55d2edae4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.321+0000 7f4b35ffb700 10 client.15363 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.321+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edae6270 0x55d2edae4cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78def28f86935035 server_cookie=55eab26a6f257103 in_seq=0 out_seq=0 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.321+0000 7f4b35ffb700 1 -- 172.21.15.66:0/1752906091 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2eda9bef0 con 0x55d2edae6270 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.321+0000 7f4b35ffb700 10 client.15363 handle_command_reply: tid=0 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 client.15363 shutdown 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 msgr2=0x55d2edada5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 0x55d2edada5a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4b48012740 tx=0x7f4b48012a50 comp rx=0 tx=0).stop 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 shutdown_connections 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edae6270 0x55d2edae4cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad1fe0 0x55d2edae1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad0790 0x55d2edada5a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1752906091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edadaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b35ffb700 0 client.15363 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 >> 172.21.15.66:0/1752906091 conn(0x55d2ed9fb790 msgr2=0x55d2edac79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 shutdown_connections 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1752906091 wait complete. 2023-12-11T19:33:29.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.322+0000 7f4b67fc7b80 20 client.15363 trim_cache size 0 max 16384 2023-12-11T19:33:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 Processor -- start 2023-12-11T19:33:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 -- start start 2023-12-11T19:33:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edac9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad10f0 0x55d2edae1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edae4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ed9c5b80 con 0x55d2edad1fe0 2023-12-11T19:33:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ed9bc390 con 0x55d2edad10f0 2023-12-11T19:33:29.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2edae38b0 con 0x55d2edad0790 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.341+0000 7f4b52b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edae4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edae4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58396/0 (socket says 172.21.15.66:58396) 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 -- 172.21.15.66:0/1814241542 learned_addr learned my addr 172.21.15.66:0/1814241542 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b5239b700 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edac9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad10f0 0x55d2edae1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 -- 172.21.15.66:0/1814241542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 msgr2=0x55d2edac9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edac9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 -- 172.21.15.66:0/1814241542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad10f0 msgr2=0x55d2edae1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad10f0 0x55d2edae1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 -- 172.21.15.66:0/1814241542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ed832c10 con 0x55d2edad1fe0 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad10f0 0x55d2edae1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:29.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edae4cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400b770 tx=0x7f4b44021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130f6e93680bc657 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b413ff700 1 -- 172.21.15.66:0/1814241542 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400e040 con 0x55d2edad1fe0 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b413ff700 1 -- 172.21.15.66:0/1814241542 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b4401eaf0 con 0x55d2edad1fe0 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.342+0000 7f4b413ff700 1 -- 172.21.15.66:0/1814241542 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4402ba50 con 0x55d2edad1fe0 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1814241542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 msgr2=0x55d2edae4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edae4cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400b770 tx=0x7f4b44021000 comp rx=0 tx=0).stop 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1814241542 shutdown_connections 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edae4cd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad0790 0x55d2edac9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/1814241542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad10f0 0x55d2edae1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1814241542 >> 172.21.15.66:0/1814241542 conn(0x55d2ed9fb790 msgr2=0x55d2edac8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1814241542 shutdown_connections 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/1814241542 wait complete. 2023-12-11T19:33:29.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.343+0000 7f4b67fc7b80 1 -- start start 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 Processor -- start 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad0790 0x55d2edae1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edae1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edaddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2edade6e0 con 0x55d2edad1fe0 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2edade850 con 0x55d2edad0790 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2edadea10 con 0x55d2edad10f0 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b67fc7b80 10 client.0 _ll_register_callbacks cb 0x55d2ed905a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b5239b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad0790 0x55d2edae1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b52b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edaddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b5239b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad0790 0x55d2edae1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:39314/0 (socket says 172.21.15.66:39314) 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.344+0000 7f4b5239b700 1 -- 172.21.15.66:0/3325258341 learned_addr learned my addr 172.21.15.66:0/3325258341 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:29.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b51b9a700 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edae1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b52b9c700 1 -- 172.21.15.66:0/3325258341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 msgr2=0x55d2edae1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edae1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b52b9c700 1 -- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad0790 msgr2=0x55d2edae1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad0790 0x55d2edae1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b52b9c700 1 -- 172.21.15.66:0/3325258341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c10a330 con 0x55d2edad1fe0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b52b9c700 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edaddff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4b44007920 tx=0x7f4b4401e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2387643b0d6dd1cc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b35ffb700 1 -- 172.21.15.66:0/3325258341 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4402bbe0 con 0x55d2edad1fe0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b3c099fb0 con 0x55d2edad1fe0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.345+0000 7f4b35ffb700 1 -- 172.21.15.66:0/3325258341 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b44025040 con 0x55d2edad1fe0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b35ffb700 1 -- 172.21.15.66:0/3325258341 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b4402b440 con 0x55d2edad1fe0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 10 client.15369 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d2ed870200 con 0x55d2edad1fe0 2023-12-11T19:33:29.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b35ffb700 1 -- 172.21.15.66:0/3325258341 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b44043020 con 0x55d2edad1fe0 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b35ffb700 1 -- 172.21.15.66:0/3325258341 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4b44034c20 con 0x55d2edad1fe0 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 10 client.15369 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 20 client.15369 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edb41100 0x55d2edb3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 4 client.15369 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.346+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d2ed9e6f80 con 0x55d2edb41100 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.347+0000 7f4b5239b700 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edb41100 0x55d2edb3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.347+0000 7f4b5239b700 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edb41100 0x55d2edb3d5a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57f6bb3aaf50565 server_cookie=4cf35a76c5a7f9 in_seq=0 out_seq=0 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.347+0000 7f4b35ffb700 10 client.15369 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b35ffb700 1 -- 172.21.15.66:0/3325258341 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d2ed9e6f80 con 0x55d2edb41100 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b35ffb700 10 client.15369 handle_command_reply: tid=0 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 client.15369 shutdown 2023-12-11T19:33:29.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 msgr2=0x55d2edaddff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edaddff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4b44007920 tx=0x7f4b4401e710 comp rx=0 tx=0).stop 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 shutdown_connections 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55d2edb41100 0x55d2edb3d5a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55d2edad0790 0x55d2edae1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2edad1fe0 0x55d2edaddff0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 --2- 172.21.15.66:0/3325258341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2edad10f0 0x55d2edae1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 >> 172.21.15.66:0/3325258341 conn(0x55d2ed9fb790 msgr2=0x55d2edae4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b35ffb700 0 client.15369 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 shutdown_connections 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 1 -- 172.21.15.66:0/3325258341 wait complete. 2023-12-11T19:33:29.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.348+0000 7f4b67fc7b80 20 client.15369 trim_cache size 0 max 16384 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "976cf7f8-985a-11ee-95a3-87774f69a715", 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24575, 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 96.868537618999994, 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 27, 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 88, 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 103.476170995 2023-12-11T19:33:29.353 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:29.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4b380a1500 msgr2=0x7f4b380a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4b380a1500 0x7f4b380a39c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4b44007f80 tx=0x7f4b440074e0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 msgr2=0x7f4b4c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:29.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 0x7f4b4c1979f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c9b0 tx=0x7f4b3c00ccc0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 shutdown_connections 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f4b380a1500 0x7f4b380a39c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b4c1015a0 0x7f4b4c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c103130 0x7f4b4c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 --2- 172.21.15.66:0/3961695699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b4c102790 0x7f4b4c1979f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 >> 172.21.15.66:0/3961695699 conn(0x7f4b4c0fcb10 msgr2=0x7f4b4c105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 shutdown_connections 2023-12-11T19:33:29.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:29.349+0000 7f4b545ff700 1 -- 172.21.15.66:0/3961695699 wait complete. 2023-12-11T19:33:29.368 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-12-11T19:33:29.392 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-12-11T19:33:29.393 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-12-11T19:33:29.393 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2023-12-11T19:33:29.393 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-11T19:33:29.393 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:29.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:29.393 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-12-11T19:33:29.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:29.439 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2023-12-11T19:33:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2023-12-11T19:33:29.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:33:29.557 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-12-11T19:33:29.557 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph fs ls 2023-12-11T19:33:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:29 smithi066 ceph-mon[97920]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:30.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:29 smithi070 ceph-mon[104044]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:30.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:29 smithi167 ceph-mon[102750]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:30.347 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.880+0000 7f04e87a9700 1 -- 172.21.15.66:0/3323647499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e01076f0 msgr2=0x7f04e0107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.880+0000 7f04e87a9700 1 --2- 172.21.15.66:0/3323647499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e01076f0 0x7f04e0107ad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009990 tx=0x7f04dc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 -- 172.21.15.66:0/3323647499 shutdown_connections 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 --2- 172.21.15.66:0/3323647499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0102040 0x7f04e010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 --2- 172.21.15.66:0/3323647499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e01076f0 0x7f04e0107ad0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 --2- 172.21.15.66:0/3323647499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04e0101680 0x7f04e0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 -- 172.21.15.66:0/3323647499 >> 172.21.15.66:0/3323647499 conn(0x7f04e00787c0 msgr2=0x7f04e0078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:31.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 -- 172.21.15.66:0/3323647499 shutdown_connections 2023-12-11T19:33:31.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.881+0000 7f04e87a9700 1 -- 172.21.15.66:0/3323647499 wait complete. 2023-12-11T19:33:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 Processor -- start 2023-12-11T19:33:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 -- start start 2023-12-11T19:33:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 0x7f04e019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04e0102040 0x7f04e019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e01076f0 0x7f04e0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:31.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0199fd0 con 0x7f04e0101680 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e019a140 con 0x7f04e0102040 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.882+0000 7f04e87a9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e019a300 con 0x7f04e01076f0 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 0x7f04e019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 0x7f04e019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58408/0 (socket says 172.21.15.66:58408) 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 -- 172.21.15.66:0/22616321 learned_addr learned my addr 172.21.15.66:0/22616321 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e5d44700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04e0102040 0x7f04e019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6d46700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e01076f0 0x7f04e0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 -- 172.21.15.66:0/22616321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e01076f0 msgr2=0x7f04e0199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e01076f0 0x7f04e0199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 -- 172.21.15.66:0/22616321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04e0102040 msgr2=0x7f04e019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04e0102040 0x7f04e019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 -- 172.21.15.66:0/22616321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc0095f0 con 0x7f04e0101680 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.883+0000 7f04e6545700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 0x7f04e019f830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f04dc006010 tx=0x7f04dc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:31.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.884+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc01d070 con 0x7f04e0101680 2023-12-11T19:33:31.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.884+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e019a580 con 0x7f04e0101680 2023-12-11T19:33:31.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.884+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04dc00fbc0 con 0x7f04e0101680 2023-12-11T19:33:31.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.884+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc022b50 con 0x7f04e0101680 2023-12-11T19:33:31.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.884+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e01a6820 con 0x7f04e0101680 2023-12-11T19:33:31.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.885+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e0104940 con 0x7f04e0101680 2023-12-11T19:33:31.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.887+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f04dc00f730 con 0x7f04e0101680 2023-12-11T19:33:31.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.889+0000 7f04d77fe700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04cc0a1480 0x7f04cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:31.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.890+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04dc0ce2d0 con 0x7f04e0101680 2023-12-11T19:33:31.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.890+0000 7f04e5d44700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04cc0a1480 0x7f04cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:31.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.890+0000 7f04e5d44700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04cc0a1480 0x7f04cc0a3940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04e019ab30 tx=0x7f04d000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:31.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:31.893+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f04dc097980 con 0x7f04e0101680 2023-12-11T19:33:32.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:31 smithi066 ceph-mon[97920]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:32.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:31 smithi070 ceph-mon[104044]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:32.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.151+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04e0102d10 con 0x7f04e0101680 2023-12-11T19:33:32.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.151+0000 7f04d77fe700 1 -- 172.21.15.66:0/22616321 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f04dc09afa0 con 0x7f04e0101680 2023-12-11T19:33:32.152 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-11T19:33:32.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04cc0a1480 msgr2=0x7f04cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:32.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04cc0a1480 0x7f04cc0a3940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04e019ab30 tx=0x7f04d000b410 comp rx=0 tx=0).stop 2023-12-11T19:33:32.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 msgr2=0x7f04e019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:32.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 0x7f04e019f830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f04dc006010 tx=0x7f04dc005720 comp rx=0 tx=0).stop 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 shutdown_connections 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04cc0a1480 0x7f04cc0a3940 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e01076f0 0x7f04e0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04e0101680 0x7f04e019f830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 --2- 172.21.15.66:0/22616321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04e0102040 0x7f04e019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 >> 172.21.15.66:0/22616321 conn(0x7f04e00787c0 msgr2=0x7f04e010cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 shutdown_connections 2023-12-11T19:33:32.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:32.154+0000 7f04e87a9700 1 -- 172.21.15.66:0/22616321 wait complete. 2023-12-11T19:33:32.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:31 smithi167 ceph-mon[102750]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:33:32.650 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-11T19:33:32.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-11T19:33:32.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2023-12-11T19:33:32.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-11T19:33:32.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:32.651 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-11T19:33:32.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-11T19:33:32.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-11T19:33:32.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-11T19:33:32.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:32.651 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2023-12-11T19:33:32.697 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2023-12-11T19:33:32.698 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41536sec preferred_lft 41536sec 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::6e70:b1a2:cd6d:66fc/64 scope link noprefixroute 2023-12-11T19:33:32.699 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-12-11T19:33:32.700 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-11T19:33:32.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:32.700 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-12-11T19:33:32.701 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-12-11T19:33:32.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2023-12-11T19:33:32.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2023-12-11T19:33:32.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2023-12-11T19:33:32.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-11T19:33:32.701 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-12-11T19:33:33.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:32 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/22616321' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T19:33:33.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:32 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/22616321' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T19:33:33.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:33.182 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-11T19:33:33.256 INFO:teuthology.orchestra.run.smithi066.stdout:1 2023-12-11T19:33:33.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:33.259 DEBUG:teuthology.orchestra.run.smithi066:> route 2023-12-11T19:33:33.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:32 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/22616321' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T19:33:33.325 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2023-12-11T19:33:33.325 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-11T19:33:33.325 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-11T19:33:33.325 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-11T19:33:33.326 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-11T19:33:33.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:33.327 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-12-11T19:33:33.327 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-12-11T19:33:33.327 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-11T19:33:33.327 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-11T19:33:33.327 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-11T19:33:33.327 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-12-11T19:33:34.036 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:33 smithi066 ceph-mon[97920]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:34.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:34.039 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-12-11T19:33:34.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:34.085 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2023-12-11T19:33:34.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:33 smithi070 ceph-mon[104044]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:34.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:34.169 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-12-11T19:33:34.169 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-12-11T19:33:34.169 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:34.169 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-11T19:33:34.169 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-12-11T19:33:34.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:33 smithi167 ceph-mon[102750]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:34.361 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-11T19:33:34.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-11T19:33:34.362 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-12-11T19:33:34.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:34.651 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-12-11T19:33:34.651 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-12-11T19:33:34.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2023-12-11T19:33:34.652 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2023-12-11T19:33:34.652 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-12-11T19:33:34.832 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2023-12-11T19:33:34.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-11T19:33:34.833 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:34.878 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-11T19:33:34.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-11T19:33:34.878 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:34.963 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2023-12-11T19:33:35.076 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2023-12-11T19:33:35.122 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/systemd 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/memory 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/perf_event 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/pids 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/freezer 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/hugetlb 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/blkio 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/devices 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/cpuset 2023-12-11T19:33:35.123 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2023-12-11T19:33:35.124 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/1a9d470b9ed8690004366fb68b87189661a3af2c2f632e54accbd74e0c0f30e5/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/0fcc612f23b35a97a6d132fb311a887faaa5f2ffd7f5bf72ade1a4da37f080a3/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/0 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/bbdf3cf00dac1d37aceb59bb9459b9f04899a674caf2009afd100a37f2978061/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/04f0615166818ad0311ecc1b0f5bf0a9c55731bd75adb5d592e6bc0048d8d82b/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b046b20c9b86d53306ce1fac810046234fee26616dc57b0bf956ffbdabeb8400/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/5493216d3f558ae237ef343f8929fc1a116c3a6cd985b4a8637bc75d8d8679d9/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/f5006198db8002a259096ce7c873a6d42f8178570086ec9db1660cd63236bba8/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/c3ffe7712013bc8120d5a75f6d70c7c4290e2b0ea24f8903407146ea0d61abe0/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/3109839714dd0cb32933d73cccb59bbc7473eedc58969b61564a6968122cd85a/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b8b13a182419376c77b7fa6e68cad1e1ac2c5475eeac85dcb25627956a441c8f/merged 2023-12-11T19:33:35.125 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2023-12-11T19:33:35.126 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:35.126 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:35.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:35.127 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2023-12-11T19:33:35.211 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-11T19:33:35.212 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106 --client_fs=cephfs) 2023-12-11T19:33:35.258 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2023-12-11T19:33:35.331 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/systemd 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-11T19:33:35.379 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/memory 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/perf_event 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/pids 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/freezer 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/hugetlb 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/blkio 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/devices 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/cpuset 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T19:33:35.380 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/1a9d470b9ed8690004366fb68b87189661a3af2c2f632e54accbd74e0c0f30e5/merged 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/0fcc612f23b35a97a6d132fb311a887faaa5f2ffd7f5bf72ade1a4da37f080a3/merged 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/0 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/bbdf3cf00dac1d37aceb59bb9459b9f04899a674caf2009afd100a37f2978061/merged 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/04f0615166818ad0311ecc1b0f5bf0a9c55731bd75adb5d592e6bc0048d8d82b/merged 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b046b20c9b86d53306ce1fac810046234fee26616dc57b0bf956ffbdabeb8400/merged 2023-12-11T19:33:35.381 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/5493216d3f558ae237ef343f8929fc1a116c3a6cd985b4a8637bc75d8d8679d9/merged 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/f5006198db8002a259096ce7c873a6d42f8178570086ec9db1660cd63236bba8/merged 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/c3ffe7712013bc8120d5a75f6d70c7c4290e2b0ea24f8903407146ea0d61abe0/merged 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/3109839714dd0cb32933d73cccb59bbc7473eedc58969b61564a6968122cd85a/merged 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b8b13a182419376c77b7fa6e68cad1e1ac2c5475eeac85dcb25627956a441c8f/merged 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:35.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:35.383 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2023-12-11T19:33:35.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:2023-12-11T19:33:35.454+0000 7efe4b63d5c0 -1 init, newargv = 0x5597e071fc30 newargc=15 2023-12-11T19:33:35.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[158721]: starting ceph client 2023-12-11T19:33:35.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[158721]: starting fuse 2023-12-11T19:33:36.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:35 smithi070 ceph-mon[104044]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:36.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:35 smithi167 ceph-mon[102750]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:36.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:35 smithi066 ceph-mon[97920]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:33:36.530 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2023-12-11T19:33:36.609 DEBUG:teuthology.orchestra.run.smithi066:> cat /proc/self/mounts | awk '{print $2}' 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/sys 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/proc 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/dev 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/security 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/shm 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/pts 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/run 2023-12-11T19:33:36.657 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/systemd 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/pstore 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/memory 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/perf_event 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/pids 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/freezer 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/hugetlb 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/blkio 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/devices 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/cgroup/cpuset 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/tracing 2023-12-11T19:33:36.658 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/config 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/ 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/selinux 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/mqueue 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/hugepages 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/kernel/debug 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/sys/fs/fuse/connections 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/nfs/rpc_pipefs 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/1000 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/ceph 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/1a9d470b9ed8690004366fb68b87189661a3af2c2f632e54accbd74e0c0f30e5/merged 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/0fcc612f23b35a97a6d132fb311a887faaa5f2ffd7f5bf72ade1a4da37f080a3/merged 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/run/user/0 2023-12-11T19:33:36.659 INFO:teuthology.orchestra.run.smithi066.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/bbdf3cf00dac1d37aceb59bb9459b9f04899a674caf2009afd100a37f2978061/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/04f0615166818ad0311ecc1b0f5bf0a9c55731bd75adb5d592e6bc0048d8d82b/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b046b20c9b86d53306ce1fac810046234fee26616dc57b0bf956ffbdabeb8400/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/5493216d3f558ae237ef343f8929fc1a116c3a6cd985b4a8637bc75d8d8679d9/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/f5006198db8002a259096ce7c873a6d42f8178570086ec9db1660cd63236bba8/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/c3ffe7712013bc8120d5a75f6d70c7c4290e2b0ea24f8903407146ea0d61abe0/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/3109839714dd0cb32933d73cccb59bbc7473eedc58969b61564a6968122cd85a/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/var/lib/containers/storage/overlay/b8b13a182419376c77b7fa6e68cad1e1ac2c5475eeac85dcb25627956a441c8f/merged 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:33:36.660 INFO:teuthology.orchestra.run.smithi066.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:36.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:36.661 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2023-12-11T19:33:36.747 INFO:teuthology.orchestra.run.smithi066.stdout:121 2023-12-11T19:33:36.748 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo stdin-killer -- python3 -c ' 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> import re 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> import os 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-11T19:33:36.748 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> return f 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-11T19:33:36.749 DEBUG:teuthology.orchestra.run.smithi066:> 2023-12-11T19:33:36.750 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.0")) 2023-12-11T19:33:36.750 DEBUG:teuthology.orchestra.run.smithi066:> ' 2023-12-11T19:33:36.936 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.0.158721.asok 2023-12-11T19:33:36.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-11T19:33:36.949 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.158721.asok 2023-12-11T19:33:36.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:36.949 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.158721.asok status 2023-12-11T19:33:37.184 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "a62fda73efba5520d409bb5a21bd8373289dda5d", 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "0", 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "158721", 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106" 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 15381, 2023-12-11T19:33:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 15381 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2497179090 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2023-12-11T19:33:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "192.168.0.1:0", 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2497179090 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.15381 192.168.0.1:0/2497179090", 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "192.168.0.1:0/2497179090", 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 5, 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 27, 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 88, 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2023-12-11T19:33:37.187 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:37.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:37.200 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2023-12-11T19:33:37.200 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:37.245 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2023-12-11T19:33:37.246 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:37.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:33:37.246 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:37.360 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-11T19:33:37.370 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-12-11T19:33:37.385 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 -- 172.21.15.66:0/1359760893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474102690 msgr2=0x7f0474102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1359760893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474102690 0x7f0474102af0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f04640099c0 tx=0x7f0464009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 -- 172.21.15.66:0/1359760893 shutdown_connections 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1359760893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04741014a0 0x7f04741018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1359760893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0474103030 0x7f04741098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1359760893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0474102690 0x7f0474102af0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 -- 172.21.15.66:0/1359760893 >> 172.21.15.66:0/1359760893 conn(0x7f04740fca10 msgr2=0x7f04740fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 -- 172.21.15.66:0/1359760893 shutdown_connections 2023-12-11T19:33:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.593+0000 7f047c3ff700 1 -- 172.21.15.66:0/1359760893 wait complete. 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.594+0000 7f047c3ff700 1 Processor -- start 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.594+0000 7f047c3ff700 1 -- start start 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047c3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 0x7f0474197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047c3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0474102690 0x7f0474197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047c3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0474103030 0x7f047419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047c3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047419ee70 con 0x7f04741014a0 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047c3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047419efb0 con 0x7f0474103030 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047c3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047419f140 con 0x7f0474102690 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047a19b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 0x7f0474197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047a19b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 0x7f0474197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50124/0 (socket says 172.21.15.66:50124) 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047a19b700 1 -- 172.21.15.66:0/1805771858 learned_addr learned my addr 172.21.15.66:0/1805771858 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047999a700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0474102690 0x7f0474197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047a99c700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0474103030 0x7f047419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047a19b700 1 -- 172.21.15.66:0/1805771858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0474102690 msgr2=0x7f0474197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.595+0000 7f047a19b700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0474102690 0x7f0474197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047a19b700 1 -- 172.21.15.66:0/1805771858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0474103030 msgr2=0x7f047419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047a19b700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0474103030 0x7f047419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047a19b700 1 -- 172.21.15.66:0/1805771858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04640095f0 con 0x7f04741014a0 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047a99c700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0474103030 0x7f047419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047a19b700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 0x7f0474197340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f047000d980 tx=0x7f047000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000bb60 con 0x7f04741014a0 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0470007620 con 0x7f04741014a0 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047419f420 con 0x7f04741014a0 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000f3c0 con 0x7f04741014a0 2023-12-11T19:33:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.596+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047419f980 con 0x7f04741014a0 2023-12-11T19:33:37.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.597+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0474108970 con 0x7f04741014a0 2023-12-11T19:33:37.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.599+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0470010460 con 0x7f04741014a0 2023-12-11T19:33:37.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.601+0000 7f046b7fe700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04600a1500 0x7f04600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:37.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.601+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04700ccfb0 con 0x7f04741014a0 2023-12-11T19:33:37.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.601+0000 7f047999a700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04600a1500 0x7f04600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:37.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.602+0000 7f047999a700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04600a1500 0x7f04600a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f046400c010 tx=0x7f0464005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:37.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.605+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f04700965d0 con 0x7f04741014a0 2023-12-11T19:33:37.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.820+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0474068230 con 0x7f04741014a0 2023-12-11T19:33:37.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.822+0000 7f046b7fe700 1 -- 172.21.15.66:0/1805771858 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f0470016090 con 0x7f04741014a0 2023-12-11T19:33:37.822 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:37.822 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04600a1500 msgr2=0x7f04600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04600a1500 0x7f04600a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f046400c010 tx=0x7f0464005fb0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 msgr2=0x7f0474197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 0x7f0474197340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f047000d980 tx=0x7f047000dd40 comp rx=0 tx=0).stop 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 shutdown_connections 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f04600a1500 0x7f04600a39c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0474103030 0x7f047419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04741014a0 0x7f0474197340 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 --2- 172.21.15.66:0/1805771858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0474102690 0x7f0474197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 >> 172.21.15.66:0/1805771858 conn(0x7f04740fca10 msgr2=0x7f0474105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 shutdown_connections 2023-12-11T19:33:37.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:37.824+0000 7f047c3ff700 1 -- 172.21.15.66:0/1805771858 wait complete. 2023-12-11T19:33:37.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:33:37.844 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 -- 172.21.15.66:0/1277458710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56181015a0 msgr2=0x7f56181019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 --2- 172.21.15.66:0/1277458710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56181015a0 0x7f56181019c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56080099c0 tx=0x7f5608009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 -- 172.21.15.66:0/1277458710 shutdown_connections 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 --2- 172.21.15.66:0/1277458710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618103130 0x7f56181099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 --2- 172.21.15.66:0/1277458710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56181015a0 0x7f56181019c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 --2- 172.21.15.66:0/1277458710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5618102790 0x7f5618102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.054+0000 7f561ddff700 1 -- 172.21.15.66:0/1277458710 >> 172.21.15.66:0/1277458710 conn(0x7f56180fcb10 msgr2=0x7f56180fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.055+0000 7f561ddff700 1 -- 172.21.15.66:0/1277458710 shutdown_connections 2023-12-11T19:33:38.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.055+0000 7f561ddff700 1 -- 172.21.15.66:0/1277458710 wait complete. 2023-12-11T19:33:38.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.055+0000 7f561ddff700 1 Processor -- start 2023-12-11T19:33:38.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 -- start start 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56181015a0 0x7f5618199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5618102790 0x7f5618199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 0x7f56181a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181a1190 con 0x7f5618103130 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181a12d0 con 0x7f5618102790 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f561ddff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56181a1460 con 0x7f56181015a0 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f5617fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 0x7f56181a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f5617fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 0x7f56181a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50132/0 (socket says 172.21.15.66:50132) 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f5617fff700 1 -- 172.21.15.66:0/3921995957 learned_addr learned my addr 172.21.15.66:0/3921995957 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f5616ffd700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5618102790 0x7f5618199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.056+0000 7f56177fe700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56181015a0 0x7f5618199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5617fff700 1 -- 172.21.15.66:0/3921995957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56181015a0 msgr2=0x7f5618199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5617fff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56181015a0 0x7f5618199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5617fff700 1 -- 172.21.15.66:0/3921995957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5618102790 msgr2=0x7f5618199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5617fff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5618102790 0x7f5618199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5617fff700 1 -- 172.21.15.66:0/3921995957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080095f0 con 0x7f5618103130 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5617fff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 0x7f56181a0bc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f561000ef30 tx=0x7f561000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610018070 con 0x7f5618103130 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56181a16e0 con 0x7f5618103130 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f561000f040 con 0x7f5618103130 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.057+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000cbc0 con 0x7f5618103130 2023-12-11T19:33:38.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.058+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181a1ba0 con 0x7f5618103130 2023-12-11T19:33:38.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.058+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5618068310 con 0x7f5618103130 2023-12-11T19:33:38.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.060+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5610007500 con 0x7f5618103130 2023-12-11T19:33:38.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.062+0000 7f5614ff9700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56000a1480 0x7f56000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.062+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f56100cc800 con 0x7f5618103130 2023-12-11T19:33:38.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.062+0000 7f56177fe700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56000a1480 0x7f56000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.063+0000 7f56177fe700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56000a1480 0x7f56000a3940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5608000c00 tx=0x7f5608003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:38.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.066+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5610095eb0 con 0x7f5618103130 2023-12-11T19:33:38.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:37 smithi070 ceph-mon[104044]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-12-11T19:33:38.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:37 smithi167 ceph-mon[102750]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-12-11T19:33:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.280+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56181a1e60 con 0x7f5618103130 2023-12-11T19:33:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.280+0000 7f5614ff9700 1 -- 172.21.15.66:0/3921995957 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56100994d0 con 0x7f5618103130 2023-12-11T19:33:38.281 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:33:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56000a1480 msgr2=0x7f56000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56000a1480 0x7f56000a3940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5608000c00 tx=0x7f5608003680 comp rx=0 tx=0).stop 2023-12-11T19:33:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 msgr2=0x7f56181a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 0x7f56181a0bc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f561000ef30 tx=0x7f561000c3a0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 shutdown_connections 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f56000a1480 0x7f56000a3940 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5618103130 0x7f56181a0bc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56181015a0 0x7f5618199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 --2- 172.21.15.66:0/3921995957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5618102790 0x7f5618199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 >> 172.21.15.66:0/3921995957 conn(0x7f56180fcb10 msgr2=0x7f5618107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 shutdown_connections 2023-12-11T19:33:38.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.283+0000 7f561ddff700 1 -- 172.21.15.66:0/3921995957 wait complete. 2023-12-11T19:33:38.300 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:37 smithi066 ceph-mon[97920]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-12-11T19:33:38.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.512+0000 7fecc29ff700 1 -- 172.21.15.66:0/1501672620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc103130 msgr2=0x7fecbc1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.512+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1501672620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc103130 0x7fecbc1099e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fecb400b0d0 tx=0x7fecb400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 -- 172.21.15.66:0/1501672620 shutdown_connections 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1501672620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecbc1015a0 0x7fecbc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1501672620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc103130 0x7fecbc1099e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1501672620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc102790 0x7fecbc102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 -- 172.21.15.66:0/1501672620 >> 172.21.15.66:0/1501672620 conn(0x7fecbc0fcb10 msgr2=0x7fecbc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:38.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 -- 172.21.15.66:0/1501672620 shutdown_connections 2023-12-11T19:33:38.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.513+0000 7fecc29ff700 1 -- 172.21.15.66:0/1501672620 wait complete. 2023-12-11T19:33:38.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 Processor -- start 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 -- start start 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc1015a0 0x7fecbc075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 0x7fecbc0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecbc103130 0x7fecbc073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0746a0 con 0x7fecbc102790 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc074810 con 0x7fecbc103130 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecc29ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1a1590 con 0x7fecbc1015a0 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecbb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 0x7fecbc0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.514+0000 7fecbb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 0x7fecbc0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50156/0 (socket says 172.21.15.66:50156) 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 -- 172.21.15.66:0/1239790517 learned_addr learned my addr 172.21.15.66:0/1239790517 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbbfff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc1015a0 0x7fecbc075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecc0f9c700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecbc103130 0x7fecbc073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 -- 172.21.15.66:0/1239790517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc1015a0 msgr2=0x7fecbc075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc1015a0 0x7fecbc075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 -- 172.21.15.66:0/1239790517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecbc103130 msgr2=0x7fecbc073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecbc103130 0x7fecbc073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 -- 172.21.15.66:0/1239790517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4009d00 con 0x7fecbc102790 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecbb7fe700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 0x7fecbc0739c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9a0 tx=0x7fecb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0009790 con 0x7fecbc102790 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc1a1820 con 0x7fecbc102790 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecb000ce40 con 0x7fecbc102790 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.515+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0018420 con 0x7fecbc102790 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.516+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc1a1d80 con 0x7fecbc102790 2023-12-11T19:33:38.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.517+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc108a20 con 0x7fecbc102790 2023-12-11T19:33:38.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.519+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fecb0010430 con 0x7fecbc102790 2023-12-11T19:33:38.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.520+0000 7fecb97fa700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feca40a1500 0x7feca40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.520+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fecb0014070 con 0x7fecbc102790 2023-12-11T19:33:38.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.520+0000 7fecbbfff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feca40a1500 0x7feca40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.521+0000 7fecbbfff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feca40a1500 0x7feca40a39c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fecbc1025f0 tx=0x7fecac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:38.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.524+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fecb00522f0 con 0x7fecbc102790 2023-12-11T19:33:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.738+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecbc108400 con 0x7fecbc102790 2023-12-11T19:33:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.739+0000 7fecb97fa700 1 -- 172.21.15.66:0/1239790517 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fecb0096430 con 0x7fecbc102790 2023-12-11T19:33:38.739 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feca40a1500 msgr2=0x7feca40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feca40a1500 0x7feca40a39c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fecbc1025f0 tx=0x7fecac009260 comp rx=0 tx=0).stop 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 msgr2=0x7fecbc0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 0x7fecbc0739c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9a0 tx=0x7fecb000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 shutdown_connections 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7feca40a1500 0x7feca40a39c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecbc103130 0x7fecbc073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecbc1015a0 0x7fecbc075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 --2- 172.21.15.66:0/1239790517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecbc102790 0x7fecbc0739c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.741+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 >> 172.21.15.66:0/1239790517 conn(0x7fecbc0fcb10 msgr2=0x7fecbc105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.742+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 shutdown_connections 2023-12-11T19:33:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.742+0000 7fecc29ff700 1 -- 172.21.15.66:0/1239790517 wait complete. 2023-12-11T19:33:38.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:33:38.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 -- 172.21.15.66:0/1375913502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0107260 msgr2=0x7fbda01096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 --2- 172.21.15.66:0/1375913502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0107260 0x7fbda01096d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009f90 tx=0x7fbd9800b330 comp rx=0 tx=0).stop 2023-12-11T19:33:38.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 -- 172.21.15.66:0/1375913502 shutdown_connections 2023-12-11T19:33:38.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 --2- 172.21.15.66:0/1375913502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 0x7fbda01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 --2- 172.21.15.66:0/1375913502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0107260 0x7fbda01096d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 --2- 172.21.15.66:0/1375913502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda0104930 0x7fbda0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.972+0000 7fbda73ff700 1 -- 172.21.15.66:0/1375913502 >> 172.21.15.66:0/1375913502 conn(0x7fbda00fb9f0 msgr2=0x7fbda00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:38.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.973+0000 7fbda73ff700 1 -- 172.21.15.66:0/1375913502 shutdown_connections 2023-12-11T19:33:38.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.973+0000 7fbda73ff700 1 -- 172.21.15.66:0/1375913502 wait complete. 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 Processor -- start 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 -- start start 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 0x7fbda0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 0x7fbda0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda0107260 0x7fbda01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01a1290 con 0x7fbda0104930 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01a13d0 con 0x7fbda0102000 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda73ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01a1560 con 0x7fbda0107260 2023-12-11T19:33:38.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda499a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 0x7fbda0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda499a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 0x7fbda0199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50188/0 (socket says 172.21.15.66:50188) 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda499a700 1 -- 172.21.15.66:0/477853859 learned_addr learned my addr 172.21.15.66:0/477853859 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.974+0000 7fbda519b700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 0x7fbda0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda599c700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda0107260 0x7fbda01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda499a700 1 -- 172.21.15.66:0/477853859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda0107260 msgr2=0x7fbda01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda499a700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda0107260 0x7fbda01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda499a700 1 -- 172.21.15.66:0/477853859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 msgr2=0x7fbda0199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda499a700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 0x7fbda0199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda499a700 1 -- 172.21.15.66:0/477853859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0095f0 con 0x7fbda0104930 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda519b700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 0x7fbda0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda499a700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 0x7fbda0199ca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9a0 tx=0x7fbd9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90009790 con 0x7fbda0104930 2023-12-11T19:33:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd9000cbd0 con 0x7fbda0104930 2023-12-11T19:33:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90018680 con 0x7fbda0104930 2023-12-11T19:33:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.975+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd98009c40 con 0x7fbda0104930 2023-12-11T19:33:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.976+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01a1b70 con 0x7fbda0104930 2023-12-11T19:33:38.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.977+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda0108de0 con 0x7fbda0104930 2023-12-11T19:33:38.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.979+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fbd9000cd40 con 0x7fbda0104930 2023-12-11T19:33:38.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.980+0000 7fbd967fc700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd8c0a1480 0x7fbd8c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:38.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.980+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd90014070 con 0x7fbda0104930 2023-12-11T19:33:38.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.980+0000 7fbda519b700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd8c0a1480 0x7fbd8c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:38.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.981+0000 7fbda519b700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd8c0a1480 0x7fbd8c0a3940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005fd0 tx=0x7fbd9c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:38.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:38.984+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbd900522f0 con 0x7fbda0104930 2023-12-11T19:33:39.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:38 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1805771858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:39.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:38 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3921995957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:39.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:38 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1239790517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:39.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:38 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1805771858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:38 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3921995957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:39.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:38 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1239790517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:39.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.195+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbda01087c0 con 0x7fbda0104930 2023-12-11T19:33:39.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.196+0000 7fbd967fc700 1 -- 172.21.15.66:0/477853859 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd900965e0 con 0x7fbda0104930 2023-12-11T19:33:39.196 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:33:39.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd8c0a1480 msgr2=0x7fbd8c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd8c0a1480 0x7fbd8c0a3940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005fd0 tx=0x7fbd9c009310 comp rx=0 tx=0).stop 2023-12-11T19:33:39.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 msgr2=0x7fbda0199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 0x7fbda0199ca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9a0 tx=0x7fbd9000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 shutdown_connections 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbd8c0a1480 0x7fbd8c0a3940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda0102000 0x7fbda0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbda0107260 0x7fbda01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 --2- 172.21.15.66:0/477853859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbda0104930 0x7fbda0199ca0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 >> 172.21.15.66:0/477853859 conn(0x7fbda00fb9f0 msgr2=0x7fbda0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 shutdown_connections 2023-12-11T19:33:39.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.198+0000 7fbda73ff700 1 -- 172.21.15.66:0/477853859 wait complete. 2023-12-11T19:33:39.217 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:39.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:38 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1805771858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:39.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:38 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3921995957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:39.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:38 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1239790517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:39.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.424+0000 7fdb8ffff700 1 -- 172.21.15.66:0/636246528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb881014a0 msgr2=0x7fdb881018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.424+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/636246528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb881014a0 0x7fdb881018c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdb840099c0 tx=0x7fdb84009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 -- 172.21.15.66:0/636246528 shutdown_connections 2023-12-11T19:33:39.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/636246528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb88103030 0x7fdb881098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/636246528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb881014a0 0x7fdb881018c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/636246528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb88102690 0x7fdb88102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 -- 172.21.15.66:0/636246528 >> 172.21.15.66:0/636246528 conn(0x7fdb880fca10 msgr2=0x7fdb880fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:39.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 -- 172.21.15.66:0/636246528 shutdown_connections 2023-12-11T19:33:39.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.425+0000 7fdb8ffff700 1 -- 172.21.15.66:0/636246528 wait complete. 2023-12-11T19:33:39.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.426+0000 7fdb8ffff700 1 Processor -- start 2023-12-11T19:33:39.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.426+0000 7fdb8ffff700 1 -- start start 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 0x7fdb881973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb88102690 0x7fdb881978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb88103030 0x7fdb8819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8ffff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8819eed0 con 0x7fdb88103030 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8ffff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8819f010 con 0x7fdb88102690 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8819f1a0 con 0x7fdb881014a0 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 0x7fdb881973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb88103030 0x7fdb8819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 0x7fdb881973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:39356/0 (socket says 172.21.15.66:39356) 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8dd9b700 1 -- 172.21.15.66:0/1238889871 learned_addr learned my addr 172.21.15.66:0/1238889871 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:39.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.427+0000 7fdb8d59a700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb88102690 0x7fdb881978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8e59c700 1 -- 172.21.15.66:0/1238889871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 msgr2=0x7fdb881973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8e59c700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 0x7fdb881973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8e59c700 1 -- 172.21.15.66:0/1238889871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb88102690 msgr2=0x7fdb881978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8e59c700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb88102690 0x7fdb881978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8e59c700 1 -- 172.21.15.66:0/1238889871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840095f0 con 0x7fdb88103030 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8dd9b700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 0x7fdb881973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:39.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8e59c700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb88103030 0x7fdb8819e900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b330 tx=0x7fdb8000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:39.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80015070 con 0x7fdb88103030 2023-12-11T19:33:39.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb88073800 con 0x7fdb88103030 2023-12-11T19:33:39.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb8000be80 con 0x7fdb88103030 2023-12-11T19:33:39.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80007bf0 con 0x7fdb88103030 2023-12-11T19:33:39.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.428+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb88073cc0 con 0x7fdb88103030 2023-12-11T19:33:39.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.429+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb88108960 con 0x7fdb88103030 2023-12-11T19:33:39.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.431+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fdb80004ad0 con 0x7fdb88103030 2023-12-11T19:33:39.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.433+0000 7fdb7effd700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb740a1480 0x7fdb740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.433+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb800d0640 con 0x7fdb88103030 2023-12-11T19:33:39.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.433+0000 7fdb8dd9b700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb740a1480 0x7fdb740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.434+0000 7fdb8dd9b700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb740a1480 0x7fdb740a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006010 tx=0x7fdb8401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:39.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.437+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdb80099c60 con 0x7fdb88103030 2023-12-11T19:33:39.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.655+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb88068230 con 0x7fdb88103030 2023-12-11T19:33:39.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.657+0000 7fdb7effd700 1 -- 172.21.15.66:0/1238889871 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7fdb8009d280 con 0x7fdb88103030 2023-12-11T19:33:39.660 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:39.661 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb740a1480 msgr2=0x7fdb740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb740a1480 0x7fdb740a3940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006010 tx=0x7fdb8401a040 comp rx=0 tx=0).stop 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb88103030 msgr2=0x7fdb8819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb88103030 0x7fdb8819e900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b330 tx=0x7fdb8000b640 comp rx=0 tx=0).stop 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 shutdown_connections 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb740a1480 0x7fdb740a3940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb88103030 0x7fdb8819e900 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb881014a0 0x7fdb881973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 --2- 172.21.15.66:0/1238889871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb88102690 0x7fdb881978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.659+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 >> 172.21.15.66:0/1238889871 conn(0x7fdb880fca10 msgr2=0x7fdb88105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.660+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 shutdown_connections 2023-12-11T19:33:39.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.660+0000 7fdb8ffff700 1 -- 172.21.15.66:0/1238889871 wait complete. 2023-12-11T19:33:39.679 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:33:39.680 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:33:39.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 -- 172.21.15.66:0/146286575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 msgr2=0x7f5e801043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 --2- 172.21.15.66:0/146286575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801043f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009990 tx=0x7f5e7c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 -- 172.21.15.66:0/146286575 shutdown_connections 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 --2- 172.21.15.66:0/146286575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e80107260 0x7f5e801096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 --2- 172.21.15.66:0/146286575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801043f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 --2- 172.21.15.66:0/146286575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 0x7f5e80106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 -- 172.21.15.66:0/146286575 >> 172.21.15.66:0/146286575 conn(0x7f5e800fb9f0 msgr2=0x7f5e800fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 -- 172.21.15.66:0/146286575 shutdown_connections 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.890+0000 7f5e879ff700 1 -- 172.21.15.66:0/146286575 wait complete. 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.891+0000 7f5e879ff700 1 Processor -- start 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.891+0000 7f5e879ff700 1 -- start start 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e879ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e879ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 0x7f5e800fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e879ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e80107260 0x7f5e801004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e879ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80100bf0 con 0x7f5e80102000 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e879ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80100d60 con 0x7f5e80104930 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e879ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e801a1540 con 0x7f5e80107260 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50222/0 (socket says 172.21.15.66:50222) 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 -- 172.21.15.66:0/1793653492 learned_addr learned my addr 172.21.15.66:0/1793653492 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e85f9c700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e80107260 0x7f5e801004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e84f9a700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 0x7f5e800fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 -- 172.21.15.66:0/1793653492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e80107260 msgr2=0x7f5e801004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e80107260 0x7f5e801004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 -- 172.21.15.66:0/1793653492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 msgr2=0x7f5e800fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:39.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 0x7f5e800fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e8579b700 1 -- 172.21.15.66:0/1793653492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0095f0 con 0x7f5e80102000 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.892+0000 7f5e84f9a700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 0x7f5e800fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.893+0000 7f5e8579b700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801018d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0098e0 tx=0x7f5e7c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.893+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01d070 con 0x7f5e80102000 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.893+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e7c004000 con 0x7f5e80102000 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.893+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c022ca0 con 0x7f5e80102000 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.893+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e801a1740 con 0x7f5e80102000 2023-12-11T19:33:39.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.893+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e801a1b60 con 0x7f5e80102000 2023-12-11T19:33:39.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.894+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e800fd570 con 0x7f5e80102000 2023-12-11T19:33:39.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.896+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5e7c022470 con 0x7f5e80102000 2023-12-11T19:33:39.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.898+0000 7f5e767fc700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e6c0a1500 0x7f5e6c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:39.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.898+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5e7c0ce1a0 con 0x7f5e80102000 2023-12-11T19:33:39.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.898+0000 7f5e84f9a700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e6c0a1500 0x7f5e6c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:39.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.899+0000 7f5e84f9a700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e6c0a1500 0x7f5e6c0a39c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007900 tx=0x7f5e70008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:39.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:39.902+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e7c097850 con 0x7f5e80102000 2023-12-11T19:33:40.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:39 smithi066 ceph-mon[97920]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:33:40.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:39 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/477853859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:40.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:39 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1238889871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:40.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:39 smithi070 ceph-mon[104044]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:33:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:39 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/477853859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:39 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1238889871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:40.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.115+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e800fcf50 con 0x7f5e80102000 2023-12-11T19:33:40.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.115+0000 7f5e767fc700 1 -- 172.21.15.66:0/1793653492 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e7c09ae70 con 0x7f5e80102000 2023-12-11T19:33:40.116 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:33:40.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e6c0a1500 msgr2=0x7f5e6c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e6c0a1500 0x7f5e6c0a39c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007900 tx=0x7f5e70008040 comp rx=0 tx=0).stop 2023-12-11T19:33:40.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 msgr2=0x7f5e801018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801018d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0098e0 tx=0x7f5e7c005f10 comp rx=0 tx=0).stop 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 shutdown_connections 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5e6c0a1500 0x7f5e6c0a39c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e80107260 0x7f5e801004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e80102000 0x7f5e801018d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 --2- 172.21.15.66:0/1793653492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e80104930 0x7f5e800fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 >> 172.21.15.66:0/1793653492 conn(0x7f5e800fb9f0 msgr2=0x7f5e80103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 shutdown_connections 2023-12-11T19:33:40.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.118+0000 7f5e879ff700 1 -- 172.21.15.66:0/1793653492 wait complete. 2023-12-11T19:33:40.137 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:33:40.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:39 smithi167 ceph-mon[102750]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:33:40.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:39 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/477853859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:40.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:39 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1238889871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:40.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/1756683762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb58103130 msgr2=0x7fbb581099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/1756683762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb58103130 0x7fbb581099e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b0d0 tx=0x7fbb5000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/1756683762 shutdown_connections 2023-12-11T19:33:40.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/1756683762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb581015a0 0x7fbb581019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/1756683762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb58103130 0x7fbb581099e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/1756683762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb58102790 0x7fbb58102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/1756683762 >> 172.21.15.66:0/1756683762 conn(0x7fbb580fcb10 msgr2=0x7fbb580fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/1756683762 shutdown_connections 2023-12-11T19:33:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.350+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/1756683762 wait complete. 2023-12-11T19:33:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.351+0000 7fbb5d5ff700 1 Processor -- start 2023-12-11T19:33:40.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.351+0000 7fbb5d5ff700 1 -- start start 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb5d5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 0x7fbb58075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb5d5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb58102790 0x7fbb580739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb5d5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb58103130 0x7fbb58073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb5d5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb580746a0 con 0x7fbb581015a0 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb5d5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58074810 con 0x7fbb58102790 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb5d5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb581a1590 con 0x7fbb58103130 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 0x7fbb58075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 0x7fbb58075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50248/0 (socket says 172.21.15.66:50248) 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 -- 172.21.15.66:0/3632588455 learned_addr learned my addr 172.21.15.66:0/3632588455 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb577fe700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb58103130 0x7fbb58073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb567fc700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb58102790 0x7fbb580739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 -- 172.21.15.66:0/3632588455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb58103130 msgr2=0x7fbb58073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb58103130 0x7fbb58073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 -- 172.21.15.66:0/3632588455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb58102790 msgr2=0x7fbb580739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb58102790 0x7fbb580739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb56ffd700 1 -- 172.21.15.66:0/3632588455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009d00 con 0x7fbb581015a0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.352+0000 7fbb567fc700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb58102790 0x7fbb580739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.353+0000 7fbb56ffd700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 0x7fbb58075330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e970 tx=0x7fbb4800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.353+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48009790 con 0x7fbb581015a0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.353+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb48004020 con 0x7fbb581015a0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.353+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb581a1820 con 0x7fbb581015a0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.353+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48003c50 con 0x7fbb581015a0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.353+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb581a1d30 con 0x7fbb581015a0 2023-12-11T19:33:40.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.354+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb58108a20 con 0x7fbb581015a0 2023-12-11T19:33:40.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.356+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fbb48004190 con 0x7fbb581015a0 2023-12-11T19:33:40.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.357+0000 7fbb3ffff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbb400a1480 0x7fbb400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.358+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbb48014070 con 0x7fbb581015a0 2023-12-11T19:33:40.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.358+0000 7fbb567fc700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbb400a1480 0x7fbb400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.358+0000 7fbb567fc700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbb400a1480 0x7fbb400a3940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009be0 tx=0x7fbb4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:40.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.361+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbb48096660 con 0x7fbb581015a0 2023-12-11T19:33:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.574+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb5810cf20 con 0x7fbb581015a0 2023-12-11T19:33:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.575+0000 7fbb3ffff700 1 -- 172.21.15.66:0/3632588455 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb48099c80 con 0x7fbb581015a0 2023-12-11T19:33:40.575 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:33:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbb400a1480 msgr2=0x7fbb400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbb400a1480 0x7fbb400a3940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009be0 tx=0x7fbb4c009260 comp rx=0 tx=0).stop 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 msgr2=0x7fbb58075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 0x7fbb58075330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e970 tx=0x7fbb4800ed30 comp rx=0 tx=0).stop 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 shutdown_connections 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbb400a1480 0x7fbb400a3940 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb58103130 0x7fbb58073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb581015a0 0x7fbb58075330 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 --2- 172.21.15.66:0/3632588455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb58102790 0x7fbb580739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 >> 172.21.15.66:0/3632588455 conn(0x7fbb580fcb10 msgr2=0x7fbb58105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 shutdown_connections 2023-12-11T19:33:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.577+0000 7fbb5d5ff700 1 -- 172.21.15.66:0/3632588455 wait complete. 2023-12-11T19:33:40.596 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:33:40.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.807+0000 7f183c9ff700 1 -- 172.21.15.66:0/3946653878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103130 msgr2=0x7f18381099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.807+0000 7f183c9ff700 1 --2- 172.21.15.66:0/3946653878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103130 0x7f18381099e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f183000b0d0 tx=0x7f183000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 -- 172.21.15.66:0/3946653878 shutdown_connections 2023-12-11T19:33:40.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 --2- 172.21.15.66:0/3946653878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18381015a0 0x7f18381019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 --2- 172.21.15.66:0/3946653878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103130 0x7f18381099e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 --2- 172.21.15.66:0/3946653878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102790 0x7f1838102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 -- 172.21.15.66:0/3946653878 >> 172.21.15.66:0/3946653878 conn(0x7f18380fcb10 msgr2=0x7f18380fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:40.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 -- 172.21.15.66:0/3946653878 shutdown_connections 2023-12-11T19:33:40.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.808+0000 7f183c9ff700 1 -- 172.21.15.66:0/3946653878 wait complete. 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 Processor -- start 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 -- start start 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 0x7f18381997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102790 0x7f1838199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1838103130 0x7f18381a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a12a0 con 0x7f18381015a0 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a13e0 con 0x7f1838103130 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183c9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a1570 con 0x7f1838102790 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 0x7f18381997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 0x7f18381997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50272/0 (socket says 172.21.15.66:50272) 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.809+0000 7f183659c700 1 -- 172.21.15.66:0/1646709396 learned_addr learned my addr 172.21.15.66:0/1646709396 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f1835d9b700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102790 0x7f1838199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f1836d9d700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1838103130 0x7f18381a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183659c700 1 -- 172.21.15.66:0/1646709396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102790 msgr2=0x7f1838199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183659c700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102790 0x7f1838199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183659c700 1 -- 172.21.15.66:0/1646709396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1838103130 msgr2=0x7f18381a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183659c700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1838103130 0x7f18381a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183659c700 1 -- 172.21.15.66:0/1646709396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830009d00 con 0x7f18381015a0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183659c700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 0x7f18381997c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f182800e970 tx=0x7f182800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828009790 con 0x7f18381015a0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f182800c7e0 con 0x7f18381015a0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18381a1850 con 0x7f18381015a0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.810+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828003bd0 con 0x7f18381015a0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.811+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18381a1d60 con 0x7f18381015a0 2023-12-11T19:33:40.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.812+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1838068310 con 0x7f18381015a0 2023-12-11T19:33:40.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.814+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f182800c950 con 0x7f18381015a0 2023-12-11T19:33:40.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.816+0000 7f18277fe700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18200a1480 0x7f18200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:40.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.816+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1828014070 con 0x7f18381015a0 2023-12-11T19:33:40.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.816+0000 7f1835d9b700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18200a1480 0x7f18200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:40.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.817+0000 7f1835d9b700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18200a1480 0x7f18200a3940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f182c005950 tx=0x7f182c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:40.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:40.820+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1828096510 con 0x7f18381015a0 2023-12-11T19:33:41.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.033+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18381a2120 con 0x7f18381015a0 2023-12-11T19:33:41.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.034+0000 7f18277fe700 1 -- 172.21.15.66:0/1646709396 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1828099b30 con 0x7f18381015a0 2023-12-11T19:33:41.034 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18200a1480 msgr2=0x7f18200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18200a1480 0x7f18200a3940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f182c005950 tx=0x7f182c00b410 comp rx=0 tx=0).stop 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 msgr2=0x7f18381997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 0x7f18381997c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f182800e970 tx=0x7f182800ec80 comp rx=0 tx=0).stop 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 shutdown_connections 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f18200a1480 0x7f18200a3940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1838103130 0x7f18381a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18381015a0 0x7f18381997c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 --2- 172.21.15.66:0/1646709396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102790 0x7f1838199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 >> 172.21.15.66:0/1646709396 conn(0x7f18380fcb10 msgr2=0x7f1838073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.036+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 shutdown_connections 2023-12-11T19:33:41.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.037+0000 7f183c9ff700 1 -- 172.21.15.66:0/1646709396 wait complete. 2023-12-11T19:33:41.053 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:40 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1793653492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:41.053 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:40 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3632588455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:41.055 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-12-11T19:33:41.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:33:41.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:40 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1793653492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:41.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:40 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3632588455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:41.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 -- 172.21.15.66:0/2074258234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 msgr2=0x7f64f81018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/2074258234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f81018c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f64e80099c0 tx=0x7f64e8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 -- 172.21.15.66:0/2074258234 shutdown_connections 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/2074258234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 0x7f64f81098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/2074258234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f81018c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/2074258234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8102690 0x7f64f8102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 -- 172.21.15.66:0/2074258234 >> 172.21.15.66:0/2074258234 conn(0x7f64f80fca10 msgr2=0x7f64f80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 -- 172.21.15.66:0/2074258234 shutdown_connections 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.263+0000 7f64fd3ff700 1 -- 172.21.15.66:0/2074258234 wait complete. 2023-12-11T19:33:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 Processor -- start 2023-12-11T19:33:41.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 -- start start 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f8199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8102690 0x7f64f8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 0x7f64f81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f81a10c0 con 0x7f64f81014a0 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f81a1200 con 0x7f64f8102690 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.265+0000 7f64fd3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f81a1390 con 0x7f64f8103030 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f8199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f8199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50292/0 (socket says 172.21.15.66:50292) 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 -- 172.21.15.66:0/4140420137 learned_addr learned my addr 172.21.15.66:0/4140420137 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64effff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8102690 0x7f64f8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f77fe700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 0x7f64f81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 -- 172.21.15.66:0/4140420137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 msgr2=0x7f64f81a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 0x7f64f81a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 -- 172.21.15.66:0/4140420137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8102690 msgr2=0x7f64f8199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8102690 0x7f64f8199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 -- 172.21.15.66:0/4140420137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80095f0 con 0x7f64f81014a0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f77fe700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 0x7f64f81a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.266+0000 7f64f6ffd700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f8199590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009f50 tx=0x7f64e8003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.267+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e801d070 con 0x7f64f81014a0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.267+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64e800bda0 con 0x7f64f81014a0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.267+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f81a1610 con 0x7f64f81014a0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.267+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8022d60 con 0x7f64f81014a0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.267+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f81a1a90 con 0x7f64f81014a0 2023-12-11T19:33:41.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.268+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f8068230 con 0x7f64f81014a0 2023-12-11T19:33:41.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:40 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1793653492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:41.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:40 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3632588455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:41.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.270+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f64e8022500 con 0x7f64f81014a0 2023-12-11T19:33:41.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.272+0000 7f64f4ff9700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e40a1500 0x7f64e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.273+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f64e80ce2c0 con 0x7f64f81014a0 2023-12-11T19:33:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.273+0000 7f64effff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e40a1500 0x7f64e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.273+0000 7f64effff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e40a1500 0x7f64e40a39c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005950 tx=0x7f64e000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:41.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.275+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f64e80979b0 con 0x7f64f81014a0 2023-12-11T19:33:41.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.540+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64f810cd10 con 0x7f64f81014a0 2023-12-11T19:33:41.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.541+0000 7f64f4ff9700 1 -- 172.21.15.66:0/4140420137 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f64e809afd0 con 0x7f64f81014a0 2023-12-11T19:33:41.541 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:41.541 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e40a1500 msgr2=0x7f64e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e40a1500 0x7f64e40a39c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005950 tx=0x7f64e000b410 comp rx=0 tx=0).stop 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 msgr2=0x7f64f8199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f8199590 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009f50 tx=0x7f64e8003a50 comp rx=0 tx=0).stop 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 shutdown_connections 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e40a1500 0x7f64e40a39c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f8103030 0x7f64f81a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014a0 0x7f64f8199590 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 --2- 172.21.15.66:0/4140420137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8102690 0x7f64f8199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 >> 172.21.15.66:0/4140420137 conn(0x7f64f80fca10 msgr2=0x7f64f8107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.543+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 shutdown_connections 2023-12-11T19:33:41.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.544+0000 7f64fd3ff700 1 -- 172.21.15.66:0/4140420137 wait complete. 2023-12-11T19:33:41.545 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:33:41.563 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-12-11T19:33:41.563 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-12-11T19:33:41.563 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T19:33:41.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/2719766311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 msgr2=0x7fb1f41018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/2719766311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f41018c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00099c0 tx=0x7fb1f0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/2719766311 shutdown_connections 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/2719766311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f4103030 0x7fb1f41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/2719766311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f41018c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/2719766311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f4102690 0x7fb1f4102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/2719766311 >> 172.21.15.66:0/2719766311 conn(0x7fb1f40fca10 msgr2=0x7fb1f40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/2719766311 shutdown_connections 2023-12-11T19:33:41.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.772+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/2719766311 wait complete. 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 Processor -- start 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 -- start start 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f4075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f4102690 0x7fb1f40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f4103030 0x7fb1f4073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f419b160 con 0x7fb1f41014a0 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4074440 con 0x7fb1f4102690 2023-12-11T19:33:41.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.773+0000 7fb1fb3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4074600 con 0x7fb1f4103030 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f4075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f4075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50302/0 (socket says 172.21.15.66:50302) 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 -- 172.21.15.66:0/921516780 learned_addr learned my addr 172.21.15.66:0/921516780 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f899a700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f4102690 0x7fb1f40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 -- 172.21.15.66:0/921516780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f4103030 msgr2=0x7fb1f4073f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f4103030 0x7fb1f4073f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 -- 172.21.15.66:0/921516780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f4102690 msgr2=0x7fb1f40739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f4102690 0x7fb1f40739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 -- 172.21.15.66:0/921516780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00095f0 con 0x7fb1f41014a0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.774+0000 7fb1f919b700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f4075330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009f50 tx=0x7fb1f0003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.775+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f001d070 con 0x7fb1f41014a0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.775+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1f000bda0 con 0x7fb1f41014a0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.775+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0022dc0 con 0x7fb1f41014a0 2023-12-11T19:33:41.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.775+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f4074820 con 0x7fb1f41014a0 2023-12-11T19:33:41.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.775+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f41a5da0 con 0x7fb1f41014a0 2023-12-11T19:33:41.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.776+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f4108920 con 0x7fb1f41014a0 2023-12-11T19:33:41.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.778+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb1f0004000 con 0x7fb1f41014a0 2023-12-11T19:33:41.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.780+0000 7fb1ea7fc700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1e00a1500 0x7fb1e00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:41.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.780+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb1f00ce150 con 0x7fb1f41014a0 2023-12-11T19:33:41.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.780+0000 7fb1f899a700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1e00a1500 0x7fb1e00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:41.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.781+0000 7fb1f899a700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1e00a1500 0x7fb1e00a39c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4005ea0 tx=0x7fb1e4005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:41.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.784+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb1f0097840 con 0x7fb1f41014a0 2023-12-11T19:33:41.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:41.995+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1f402d3d0 con 0x7fb1e00a1500 2023-12-11T19:33:42.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.007+0000 7fb1ea7fc700 1 -- 172.21.15.66:0/921516780 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343469 (secure 0 0 0) 0x7fb1f402d3d0 con 0x7fb1e00a1500 2023-12-11T19:33:42.012 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1e00a1500 msgr2=0x7fb1e00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1e00a1500 0x7fb1e00a39c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4005ea0 tx=0x7fb1e4005e30 comp rx=0 tx=0).stop 2023-12-11T19:33:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 msgr2=0x7fb1f4075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f4075330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009f50 tx=0x7fb1f0003b00 comp rx=0 tx=0).stop 2023-12-11T19:33:42.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 shutdown_connections 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1e00a1500 0x7fb1e00a39c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f4103030 0x7fb1f4073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1f41014a0 0x7fb1f4075330 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 --2- 172.21.15.66:0/921516780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1f4102690 0x7fb1f40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 >> 172.21.15.66:0/921516780 conn(0x7fb1f40fca10 msgr2=0x7fb1f4105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 shutdown_connections 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.012+0000 7fb1fb3ff700 1 -- 172.21.15.66:0/921516780 wait complete. 2023-12-11T19:33:42.016 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:33:42.031 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2023-12-11T19:33:40.311697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660844,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1161,"num_read_kb":712,"num_write":436,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1183040,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335976,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605848,"statfs":{"total":1151940427776,"available":1151596388352,"internally_reserved":0,"allocated":12066816,"data_stored":7483027,"data_compressed":183741,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2490,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":14,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003021"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343455+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:33:10.343455+0000","last_peered":"2023-12-11T19:33:10.343455+0000","last_clean":"2023-12-11T19:33:10.343455+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:33:10.343455+0000","last_undegraded":"2023-12-11T19:33:10.343455+0000","last_fullsized":"2023-12-11T19:33:10.343455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.334250+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:33:10.334250+0000","last_peered":"2023-12-11T19:33:10.334250+0000","last_clean":"2023-12-11T19:33:10.334250+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:33:10.334250+0000","last_undegraded":"2023-12-11T19:33:10.334250+0000","last_fullsized":"2023-12-11T19:33:10.334250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340675+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:33:10.340675+0000","last_peered":"2023-12-11T19:33:10.340675+0000","last_clean":"2023-12-11T19:33:10.340675+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:33:10.340675+0000","last_undegraded":"2023-12-11T19:33:10.340675+0000","last_fullsized":"2023-12-11T19:33:10.340675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.336763+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:33:10.336763+0000","last_peered":"2023-12-11T19:33:10.336763+0000","last_clean":"2023-12-11T19:33:10.336763+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:33:10.336763+0000","last_undegraded":"2023-12-11T19:33:10.336763+0000","last_fullsized":"2023-12-11T19:33:10.336763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324900+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:33:10.324900+0000","last_peered":"2023-12-11T19:33:10.324900+0000","last_clean":"2023-12-11T19:33:10.324900+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:33:10.324900+0000","last_undegraded":"2023-12-11T19:33:10.324900+0000","last_fullsized":"2023-12-11T19:33:10.324900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324677+0000","last_change":"2023-12-11T19:32:11.585539+0000","last_active":"2023-12-11T19:33:10.324677+0000","last_peered":"2023-12-11T19:33:10.324677+0000","last_clean":"2023-12-11T19:33:10.324677+0000","last_became_active":"2023-12-11T19:32:11.585087+0000","last_became_peered":"2023-12-11T19:32:11.585087+0000","last_unstale":"2023-12-11T19:33:10.324677+0000","last_undegraded":"2023-12-11T19:33:10.324677+0000","last_fullsized":"2023-12-11T19:33:10.324677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:23.076166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332857+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:33:10.332857+0000","last_peered":"2023-12-11T19:33:10.332857+0000","last_clean":"2023-12-11T19:33:10.332857+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:33:10.332857+0000","last_undegraded":"2023-12-11T19:33:10.332857+0000","last_fullsized":"2023-12-11T19:33:10.332857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341918+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:33:10.341918+0000","last_peered":"2023-12-11T19:33:10.341918+0000","last_clean":"2023-12-11T19:33:10.341918+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:33:10.341918+0000","last_undegraded":"2023-12-11T19:33:10.341918+0000","last_fullsized":"2023-12-11T19:33:10.341918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343323+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:33:10.343323+0000","last_peered":"2023-12-11T19:33:10.343323+0000","last_clean":"2023-12-11T19:33:10.343323+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:33:10.343323+0000","last_undegraded":"2023-12-11T19:33:10.343323+0000","last_fullsized":"2023-12-11T19:33:10.343323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339811+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:33:10.339811+0000","last_peered":"2023-12-11T19:33:10.339811+0000","last_clean":"2023-12-11T19:33:10.339811+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:33:10.339811+0000","last_undegraded":"2023-12-11T19:33:10.339811+0000","last_fullsized":"2023-12-11T19:33:10.339811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339748+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:33:10.339748+0000","last_peered":"2023-12-11T19:33:10.339748+0000","last_clean":"2023-12-11T19:33:10.339748+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:33:10.339748+0000","last_undegraded":"2023-12-11T19:33:10.339748+0000","last_fullsized":"2023-12-11T19:33:10.339748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331280+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:33:10.331280+0000","last_peered":"2023-12-11T19:33:10.331280+0000","last_clean":"2023-12-11T19:33:10.331280+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:33:10.331280+0000","last_undegraded":"2023-12-11T19:33:10.331280+0000","last_fullsized":"2023-12-11T19:33:10.331280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337449+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:33:10.337449+0000","last_peered":"2023-12-11T19:33:10.337449+0000","last_clean":"2023-12-11T19:33:10.337449+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:33:10.337449+0000","last_undegraded":"2023-12-11T19:33:10.337449+0000","last_fullsized":"2023-12-11T19:33:10.337449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.334371+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:33:10.334371+0000","last_peered":"2023-12-11T19:33:10.334371+0000","last_clean":"2023-12-11T19:33:10.334371+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:33:10.334371+0000","last_undegraded":"2023-12-11T19:33:10.334371+0000","last_fullsized":"2023-12-11T19:33:10.334371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341068+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:33:10.341068+0000","last_peered":"2023-12-11T19:33:10.341068+0000","last_clean":"2023-12-11T19:33:10.341068+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:33:10.341068+0000","last_undegraded":"2023-12-11T19:33:10.341068+0000","last_fullsized":"2023-12-11T19:33:10.341068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344589+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:33:10.344589+0000","last_peered":"2023-12-11T19:33:10.344589+0000","last_clean":"2023-12-11T19:33:10.344589+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:33:10.344589+0000","last_undegraded":"2023-12-11T19:33:10.344589+0000","last_fullsized":"2023-12-11T19:33:10.344589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343527+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:33:10.343527+0000","last_peered":"2023-12-11T19:33:10.343527+0000","last_clean":"2023-12-11T19:33:10.343527+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:33:10.343527+0000","last_undegraded":"2023-12-11T19:33:10.343527+0000","last_fullsized":"2023-12-11T19:33:10.343527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333632+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:33:10.333632+0000","last_peered":"2023-12-11T19:33:10.333632+0000","last_clean":"2023-12-11T19:33:10.333632+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:33:10.333632+0000","last_undegraded":"2023-12-11T19:33:10.333632+0000","last_fullsized":"2023-12-11T19:33:10.333632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324560+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:33:10.324560+0000","last_peered":"2023-12-11T19:33:10.324560+0000","last_clean":"2023-12-11T19:33:10.324560+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:33:10.324560+0000","last_undegraded":"2023-12-11T19:33:10.324560+0000","last_fullsized":"2023-12-11T19:33:10.324560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:37.707150+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:33:37.707150+0000","last_peered":"2023-12-11T19:33:37.707150+0000","last_clean":"2023-12-11T19:33:37.707150+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:33:37.707150+0000","last_undegraded":"2023-12-11T19:33:37.707150+0000","last_fullsized":"2023-12-11T19:33:37.707150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.328995+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:33:10.328995+0000","last_peered":"2023-12-11T19:33:10.328995+0000","last_clean":"2023-12-11T19:33:10.328995+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:33:10.328995+0000","last_undegraded":"2023-12-11T19:33:10.328995+0000","last_fullsized":"2023-12-11T19:33:10.328995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586855+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:32:10.586855+0000","last_peered":"2023-12-11T19:32:10.586855+0000","last_clean":"2023-12-11T19:32:10.586855+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:32:10.586855+0000","last_undegraded":"2023-12-11T19:32:10.586855+0000","last_fullsized":"2023-12-11T19:32:10.586855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344983+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:33:10.344983+0000","last_peered":"2023-12-11T19:33:10.344983+0000","last_clean":"2023-12-11T19:33:10.344983+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:33:10.344983+0000","last_undegraded":"2023-12-11T19:33:10.344983+0000","last_fullsized":"2023-12-11T19:33:10.344983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585009+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:32:10.585009+0000","last_peered":"2023-12-11T19:32:10.585009+0000","last_clean":"2023-12-11T19:32:10.585009+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:32:10.585009+0000","last_undegraded":"2023-12-11T19:32:10.585009+0000","last_fullsized":"2023-12-11T19:32:10.585009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343252+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:33:10.343252+0000","last_peered":"2023-12-11T19:33:10.343252+0000","last_clean":"2023-12-11T19:33:10.343252+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:33:10.343252+0000","last_undegraded":"2023-12-11T19:33:10.343252+0000","last_fullsized":"2023-12-11T19:33:10.343252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584594+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:32:10.584594+0000","last_peered":"2023-12-11T19:32:10.584594+0000","last_clean":"2023-12-11T19:32:10.584594+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:32:10.584594+0000","last_undegraded":"2023-12-11T19:32:10.584594+0000","last_fullsized":"2023-12-11T19:32:10.584594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339985+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:33:10.339985+0000","last_peered":"2023-12-11T19:33:10.339985+0000","last_clean":"2023-12-11T19:33:10.339985+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:33:10.339985+0000","last_undegraded":"2023-12-11T19:33:10.339985+0000","last_fullsized":"2023-12-11T19:33:10.339985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:37.733788+0000","last_change":"2023-12-11T19:32:11.601552+0000","last_active":"2023-12-11T19:33:37.733788+0000","last_peered":"2023-12-11T19:33:37.733788+0000","last_clean":"2023-12-11T19:33:37.733788+0000","last_became_active":"2023-12-11T19:32:11.593828+0000","last_became_peered":"2023-12-11T19:32:11.593828+0000","last_unstale":"2023-12-11T19:33:37.733788+0000","last_undegraded":"2023-12-11T19:33:37.733788+0000","last_fullsized":"2023-12-11T19:33:37.733788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:53.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1650,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333342+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:33:10.333342+0000","last_peered":"2023-12-11T19:33:10.333342+0000","last_clean":"2023-12-11T19:33:10.333342+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:33:10.333342+0000","last_undegraded":"2023-12-11T19:33:10.333342+0000","last_fullsized":"2023-12-11T19:33:10.333342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586092+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:32:10.586092+0000","last_peered":"2023-12-11T19:32:10.586092+0000","last_clean":"2023-12-11T19:32:10.586092+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:32:10.586092+0000","last_undegraded":"2023-12-11T19:32:10.586092+0000","last_fullsized":"2023-12-11T19:32:10.586092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.338447+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:33:10.338447+0000","last_peered":"2023-12-11T19:33:10.338447+0000","last_clean":"2023-12-11T19:33:10.338447+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:33:10.338447+0000","last_undegraded":"2023-12-11T19:33:10.338447+0000","last_fullsized":"2023-12-11T19:33:10.338447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581195+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:32:10.581195+0000","last_peered":"2023-12-11T19:32:10.581195+0000","last_clean":"2023-12-11T19:32:10.581195+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:32:10.581195+0000","last_undegraded":"2023-12-11T19:32:10.581195+0000","last_fullsized":"2023-12-11T19:32:10.581195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337715+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:33:10.337715+0000","last_peered":"2023-12-11T19:33:10.337715+0000","last_clean":"2023-12-11T19:33:10.337715+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:33:10.337715+0000","last_undegraded":"2023-12-11T19:33:10.337715+0000","last_fullsized":"2023-12-11T19:33:10.337715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581346+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:32:10.581346+0000","last_peered":"2023-12-11T19:32:10.581346+0000","last_clean":"2023-12-11T19:32:10.581346+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:32:10.581346+0000","last_undegraded":"2023-12-11T19:32:10.581346+0000","last_fullsized":"2023-12-11T19:32:10.581346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342744+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:33:10.342744+0000","last_peered":"2023-12-11T19:33:10.342744+0000","last_clean":"2023-12-11T19:33:10.342744+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:33:10.342744+0000","last_undegraded":"2023-12-11T19:33:10.342744+0000","last_fullsized":"2023-12-11T19:33:10.342744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":227,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.732269+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:33:39.732269+0000","last_peered":"2023-12-11T19:33:39.732269+0000","last_clean":"2023-12-11T19:33:39.732269+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:33:39.732269+0000","last_undegraded":"2023-12-11T19:33:39.732269+0000","last_fullsized":"2023-12-11T19:33:39.732269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340936+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:33:10.340936+0000","last_peered":"2023-12-11T19:33:10.340936+0000","last_clean":"2023-12-11T19:33:10.340936+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:33:10.340936+0000","last_undegraded":"2023-12-11T19:33:10.340936+0000","last_fullsized":"2023-12-11T19:33:10.340936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578707+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:32:10.578707+0000","last_peered":"2023-12-11T19:32:10.578707+0000","last_clean":"2023-12-11T19:32:10.578707+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:32:10.578707+0000","last_undegraded":"2023-12-11T19:32:10.578707+0000","last_fullsized":"2023-12-11T19:32:10.578707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340428+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:33:10.340428+0000","last_peered":"2023-12-11T19:33:10.340428+0000","last_clean":"2023-12-11T19:33:10.340428+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:33:10.340428+0000","last_undegraded":"2023-12-11T19:33:10.340428+0000","last_fullsized":"2023-12-11T19:33:10.340428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581870+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:32:10.581870+0000","last_peered":"2023-12-11T19:32:10.581870+0000","last_clean":"2023-12-11T19:32:10.581870+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:32:10.581870+0000","last_undegraded":"2023-12-11T19:32:10.581870+0000","last_fullsized":"2023-12-11T19:32:10.581870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344002+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:33:10.344002+0000","last_peered":"2023-12-11T19:33:10.344002+0000","last_clean":"2023-12-11T19:33:10.344002+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:33:10.344002+0000","last_undegraded":"2023-12-11T19:33:10.344002+0000","last_fullsized":"2023-12-11T19:33:10.344002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":267,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.658525+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:33:39.658525+0000","last_peered":"2023-12-11T19:33:39.658525+0000","last_clean":"2023-12-11T19:33:39.658525+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:33:39.658525+0000","last_undegraded":"2023-12-11T19:33:39.658525+0000","last_fullsized":"2023-12-11T19:33:39.658525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60756,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":168,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333830+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:33:10.333830+0000","last_peered":"2023-12-11T19:33:10.333830+0000","last_clean":"2023-12-11T19:33:10.333830+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:33:10.333830+0000","last_undegraded":"2023-12-11T19:33:10.333830+0000","last_fullsized":"2023-12-11T19:33:10.333830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.587382+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:32:10.587382+0000","last_peered":"2023-12-11T19:32:10.587382+0000","last_clean":"2023-12-11T19:32:10.587382+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:32:10.587382+0000","last_undegraded":"2023-12-11T19:32:10.587382+0000","last_fullsized":"2023-12-11T19:32:10.587382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337785+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:33:10.337785+0000","last_peered":"2023-12-11T19:33:10.337785+0000","last_clean":"2023-12-11T19:33:10.337785+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:33:10.337785+0000","last_undegraded":"2023-12-11T19:33:10.337785+0000","last_fullsized":"2023-12-11T19:33:10.337785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581282+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:32:10.581282+0000","last_peered":"2023-12-11T19:32:10.581282+0000","last_clean":"2023-12-11T19:32:10.581282+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:32:10.581282+0000","last_undegraded":"2023-12-11T19:32:10.581282+0000","last_fullsized":"2023-12-11T19:32:10.581282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344461+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:33:10.344461+0000","last_peered":"2023-12-11T19:33:10.344461+0000","last_clean":"2023-12-11T19:33:10.344461+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:33:10.344461+0000","last_undegraded":"2023-12-11T19:33:10.344461+0000","last_fullsized":"2023-12-11T19:33:10.344461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'8","reported_seq":126,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:36.732277+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:33:36.732277+0000","last_peered":"2023-12-11T19:33:36.732277+0000","last_clean":"2023-12-11T19:33:36.732277+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:33:36.732277+0000","last_undegraded":"2023-12-11T19:33:36.732277+0000","last_fullsized":"2023-12-11T19:33:36.732277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1616,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341155+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:33:10.341155+0000","last_peered":"2023-12-11T19:33:10.341155+0000","last_clean":"2023-12-11T19:33:10.341155+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:33:10.341155+0000","last_undegraded":"2023-12-11T19:33:10.341155+0000","last_fullsized":"2023-12-11T19:33:10.341155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583186+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:32:10.583186+0000","last_peered":"2023-12-11T19:32:10.583186+0000","last_clean":"2023-12-11T19:32:10.583186+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:32:10.583186+0000","last_undegraded":"2023-12-11T19:32:10.583186+0000","last_fullsized":"2023-12-11T19:32:10.583186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339499+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:33:10.339499+0000","last_peered":"2023-12-11T19:33:10.339499+0000","last_clean":"2023-12-11T19:33:10.339499+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:33:10.339499+0000","last_undegraded":"2023-12-11T19:33:10.339499+0000","last_fullsized":"2023-12-11T19:33:10.339499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585021+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:32:10.585021+0000","last_peered":"2023-12-11T19:32:10.585021+0000","last_clean":"2023-12-11T19:32:10.585021+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:32:10.585021+0000","last_undegraded":"2023-12-11T19:32:10.585021+0000","last_fullsized":"2023-12-11T19:32:10.585021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332391+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:33:10.332391+0000","last_peered":"2023-12-11T19:33:10.332391+0000","last_clean":"2023-12-11T19:33:10.332391+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:33:10.332391+0000","last_undegraded":"2023-12-11T19:33:10.332391+0000","last_fullsized":"2023-12-11T19:33:10.332391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583392+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:32:10.583392+0000","last_peered":"2023-12-11T19:32:10.583392+0000","last_clean":"2023-12-11T19:32:10.583392+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:32:10.583392+0000","last_undegraded":"2023-12-11T19:32:10.583392+0000","last_fullsized":"2023-12-11T19:32:10.583392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324798+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:33:10.324798+0000","last_peered":"2023-12-11T19:33:10.324798+0000","last_clean":"2023-12-11T19:33:10.324798+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:33:10.324798+0000","last_undegraded":"2023-12-11T19:33:10.324798+0000","last_fullsized":"2023-12-11T19:33:10.324798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:36.706343+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:33:36.706343+0000","last_peered":"2023-12-11T19:33:36.706343+0000","last_clean":"2023-12-11T19:33:36.706343+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:33:36.706343+0000","last_undegraded":"2023-12-11T19:33:36.706343+0000","last_fullsized":"2023-12-11T19:33:36.706343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1086,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341698+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:33:10.341698+0000","last_peered":"2023-12-11T19:33:10.341698+0000","last_clean":"2023-12-11T19:33:10.341698+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:33:10.341698+0000","last_undegraded":"2023-12-11T19:33:10.341698+0000","last_fullsized":"2023-12-11T19:33:10.341698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.317738+0000","last_change":"2023-12-11T19:32:11.594139+0000","last_active":"2023-12-11T19:33:10.317738+0000","last_peered":"2023-12-11T19:33:10.317738+0000","last_clean":"2023-12-11T19:33:10.317738+0000","last_became_active":"2023-12-11T19:32:11.593876+0000","last_became_peered":"2023-12-11T19:32:11.593876+0000","last_unstale":"2023-12-11T19:33:10.317738+0000","last_undegraded":"2023-12-11T19:33:10.317738+0000","last_fullsized":"2023-12-11T19:33:10.317738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:22:15.243111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:35.470217+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:33:35.470217+0000","last_peered":"2023-12-11T19:33:35.470217+0000","last_clean":"2023-12-11T19:33:35.470217+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:33:35.470217+0000","last_undegraded":"2023-12-11T19:33:35.470217+0000","last_fullsized":"2023-12-11T19:33:35.470217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341374+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:33:10.341374+0000","last_peered":"2023-12-11T19:33:10.341374+0000","last_clean":"2023-12-11T19:33:10.341374+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:33:10.341374+0000","last_undegraded":"2023-12-11T19:33:10.341374+0000","last_fullsized":"2023-12-11T19:33:10.341374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582019+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:32:10.582019+0000","last_peered":"2023-12-11T19:32:10.582019+0000","last_clean":"2023-12-11T19:32:10.582019+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:32:10.582019+0000","last_undegraded":"2023-12-11T19:32:10.582019+0000","last_fullsized":"2023-12-11T19:32:10.582019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":315,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.582905+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:32:11.582905+0000","last_peered":"2023-12-11T19:32:11.582905+0000","last_clean":"2023-12-11T19:32:11.582905+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:32:11.582905+0000","last_undegraded":"2023-12-11T19:32:11.582905+0000","last_fullsized":"2023-12-11T19:32:11.582905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.325546+0000","last_change":"2023-12-11T19:32:11.590972+0000","last_active":"2023-12-11T19:33:10.325546+0000","last_peered":"2023-12-11T19:33:10.325546+0000","last_clean":"2023-12-11T19:33:10.325546+0000","last_became_active":"2023-12-11T19:32:11.590564+0000","last_became_peered":"2023-12-11T19:32:11.590564+0000","last_unstale":"2023-12-11T19:33:10.325546+0000","last_undegraded":"2023-12-11T19:33:10.325546+0000","last_fullsized":"2023-12-11T19:33:10.325546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:48:35.323725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578698+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:32:10.578698+0000","last_peered":"2023-12-11T19:32:10.578698+0000","last_clean":"2023-12-11T19:32:10.578698+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:32:10.578698+0000","last_undegraded":"2023-12-11T19:32:10.578698+0000","last_fullsized":"2023-12-11T19:32:10.578698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584892+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:32:10.584892+0000","last_peered":"2023-12-11T19:32:10.584892+0000","last_clean":"2023-12-11T19:32:10.584892+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:32:10.584892+0000","last_undegraded":"2023-12-11T19:32:10.584892+0000","last_fullsized":"2023-12-11T19:32:10.584892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339768+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:33:10.339768+0000","last_peered":"2023-12-11T19:33:10.339768+0000","last_clean":"2023-12-11T19:33:10.339768+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:33:10.339768+0000","last_undegraded":"2023-12-11T19:33:10.339768+0000","last_fullsized":"2023-12-11T19:33:10.339768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.878510+0000","last_change":"2023-12-11T19:32:11.599025+0000","last_active":"2023-12-11T19:33:10.878510+0000","last_peered":"2023-12-11T19:33:10.878510+0000","last_clean":"2023-12-11T19:33:10.878510+0000","last_became_active":"2023-12-11T19:32:11.587530+0000","last_became_peered":"2023-12-11T19:32:11.587530+0000","last_unstale":"2023-12-11T19:33:10.878510+0000","last_undegraded":"2023-12-11T19:33:10.878510+0000","last_fullsized":"2023-12-11T19:33:10.878510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:08:05.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339794+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:33:10.339794+0000","last_peered":"2023-12-11T19:33:10.339794+0000","last_clean":"2023-12-11T19:33:10.339794+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:33:10.339794+0000","last_undegraded":"2023-12-11T19:33:10.339794+0000","last_fullsized":"2023-12-11T19:33:10.339794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581975+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:32:10.581975+0000","last_peered":"2023-12-11T19:32:10.581975+0000","last_clean":"2023-12-11T19:32:10.581975+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:32:10.581975+0000","last_undegraded":"2023-12-11T19:32:10.581975+0000","last_fullsized":"2023-12-11T19:32:10.581975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342950+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:33:10.342950+0000","last_peered":"2023-12-11T19:33:10.342950+0000","last_clean":"2023-12-11T19:33:10.342950+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:33:10.342950+0000","last_undegraded":"2023-12-11T19:33:10.342950+0000","last_fullsized":"2023-12-11T19:33:10.342950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581192+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:32:10.581192+0000","last_peered":"2023-12-11T19:32:10.581192+0000","last_clean":"2023-12-11T19:32:10.581192+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:32:10.581192+0000","last_undegraded":"2023-12-11T19:32:10.581192+0000","last_fullsized":"2023-12-11T19:32:10.581192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.287117+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:33:10.287117+0000","last_peered":"2023-12-11T19:33:10.287117+0000","last_clean":"2023-12-11T19:33:10.287117+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:33:10.287117+0000","last_undegraded":"2023-12-11T19:33:10.287117+0000","last_fullsized":"2023-12-11T19:33:10.287117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585191+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:32:10.585191+0000","last_peered":"2023-12-11T19:32:10.585191+0000","last_clean":"2023-12-11T19:32:10.585191+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:32:10.585191+0000","last_undegraded":"2023-12-11T19:32:10.585191+0000","last_fullsized":"2023-12-11T19:32:10.585191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339172+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:33:10.339172+0000","last_peered":"2023-12-11T19:33:10.339172+0000","last_clean":"2023-12-11T19:33:10.339172+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:33:10.339172+0000","last_undegraded":"2023-12-11T19:33:10.339172+0000","last_fullsized":"2023-12-11T19:33:10.339172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.588890+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:32:11.588890+0000","last_peered":"2023-12-11T19:32:11.588890+0000","last_clean":"2023-12-11T19:32:11.588890+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:32:11.588890+0000","last_undegraded":"2023-12-11T19:32:11.588890+0000","last_fullsized":"2023-12-11T19:32:11.588890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342653+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:33:10.342653+0000","last_peered":"2023-12-11T19:33:10.342653+0000","last_clean":"2023-12-11T19:33:10.342653+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:33:10.342653+0000","last_undegraded":"2023-12-11T19:33:10.342653+0000","last_fullsized":"2023-12-11T19:33:10.342653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599949+0000","last_change":"2023-12-11T19:32:11.599116+0000","last_active":"2023-12-11T19:32:11.599949+0000","last_peered":"2023-12-11T19:32:11.599949+0000","last_clean":"2023-12-11T19:32:11.599949+0000","last_became_active":"2023-12-11T19:32:11.592106+0000","last_became_peered":"2023-12-11T19:32:11.592106+0000","last_unstale":"2023-12-11T19:32:11.599949+0000","last_undegraded":"2023-12-11T19:32:11.599949+0000","last_fullsized":"2023-12-11T19:32:11.599949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:23.285224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344074+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:33:10.344074+0000","last_peered":"2023-12-11T19:33:10.344074+0000","last_clean":"2023-12-11T19:33:10.344074+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:33:10.344074+0000","last_undegraded":"2023-12-11T19:33:10.344074+0000","last_fullsized":"2023-12-11T19:33:10.344074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:35.725415+0000","last_change":"2023-12-11T19:32:11.603249+0000","last_active":"2023-12-11T19:33:35.725415+0000","last_peered":"2023-12-11T19:33:35.725415+0000","last_clean":"2023-12-11T19:33:35.725415+0000","last_became_active":"2023-12-11T19:32:11.587129+0000","last_became_peered":"2023-12-11T19:32:11.587129+0000","last_unstale":"2023-12-11T19:33:35.725415+0000","last_undegraded":"2023-12-11T19:33:35.725415+0000","last_fullsized":"2023-12-11T19:33:35.725415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:28:44.762360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1668,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343509+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:33:10.343509+0000","last_peered":"2023-12-11T19:33:10.343509+0000","last_clean":"2023-12-11T19:33:10.343509+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:33:10.343509+0000","last_undegraded":"2023-12-11T19:33:10.343509+0000","last_fullsized":"2023-12-11T19:33:10.343509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581433+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:32:10.581433+0000","last_peered":"2023-12-11T19:32:10.581433+0000","last_clean":"2023-12-11T19:32:10.581433+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:32:10.581433+0000","last_undegraded":"2023-12-11T19:32:10.581433+0000","last_fullsized":"2023-12-11T19:32:10.581433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333570+0000","last_change":"2023-12-11T19:32:11.596576+0000","last_active":"2023-12-11T19:33:10.333570+0000","last_peered":"2023-12-11T19:33:10.333570+0000","last_clean":"2023-12-11T19:33:10.333570+0000","last_became_active":"2023-12-11T19:32:11.592679+0000","last_became_peered":"2023-12-11T19:32:11.592679+0000","last_unstale":"2023-12-11T19:33:10.333570+0000","last_undegraded":"2023-12-11T19:33:10.333570+0000","last_fullsized":"2023-12-11T19:33:10.333570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:26:20.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581645+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:32:10.581645+0000","last_peered":"2023-12-11T19:32:10.581645+0000","last_clean":"2023-12-11T19:32:10.581645+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:32:10.581645+0000","last_undegraded":"2023-12-11T19:32:10.581645+0000","last_fullsized":"2023-12-11T19:32:10.581645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331611+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:33:10.331611+0000","last_peered":"2023-12-11T19:33:10.331611+0000","last_clean":"2023-12-11T19:33:10.331611+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:33:10.331611+0000","last_undegraded":"2023-12-11T19:33:10.331611+0000","last_fullsized":"2023-12-11T19:33:10.331611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.335239+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:33:10.335239+0000","last_peered":"2023-12-11T19:33:10.335239+0000","last_clean":"2023-12-11T19:33:10.335239+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:33:10.335239+0000","last_undegraded":"2023-12-11T19:33:10.335239+0000","last_fullsized":"2023-12-11T19:33:10.335239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333471+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:33:10.333471+0000","last_peered":"2023-12-11T19:33:10.333471+0000","last_clean":"2023-12-11T19:33:10.333471+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:33:10.333471+0000","last_undegraded":"2023-12-11T19:33:10.333471+0000","last_fullsized":"2023-12-11T19:33:10.333471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332291+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:33:10.332291+0000","last_peered":"2023-12-11T19:33:10.332291+0000","last_clean":"2023-12-11T19:33:10.332291+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:33:10.332291+0000","last_undegraded":"2023-12-11T19:33:10.332291+0000","last_fullsized":"2023-12-11T19:33:10.332291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.336523+0000","last_change":"2023-12-11T19:32:11.586739+0000","last_active":"2023-12-11T19:33:10.336523+0000","last_peered":"2023-12-11T19:33:10.336523+0000","last_clean":"2023-12-11T19:33:10.336523+0000","last_became_active":"2023-12-11T19:32:11.586457+0000","last_became_peered":"2023-12-11T19:32:11.586457+0000","last_unstale":"2023-12-11T19:33:10.336523+0000","last_undegraded":"2023-12-11T19:33:10.336523+0000","last_fullsized":"2023-12-11T19:33:10.336523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:50.986234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341946+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:33:10.341946+0000","last_peered":"2023-12-11T19:33:10.341946+0000","last_clean":"2023-12-11T19:33:10.341946+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:33:10.341946+0000","last_undegraded":"2023-12-11T19:33:10.341946+0000","last_fullsized":"2023-12-11T19:33:10.341946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333225+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:33:10.333225+0000","last_peered":"2023-12-11T19:33:10.333225+0000","last_clean":"2023-12-11T19:33:10.333225+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:33:10.333225+0000","last_undegraded":"2023-12-11T19:33:10.333225+0000","last_fullsized":"2023-12-11T19:33:10.333225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339819+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:33:10.339819+0000","last_peered":"2023-12-11T19:33:10.339819+0000","last_clean":"2023-12-11T19:33:10.339819+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:33:10.339819+0000","last_undegraded":"2023-12-11T19:33:10.339819+0000","last_fullsized":"2023-12-11T19:33:10.339819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.319673+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:33:10.319673+0000","last_peered":"2023-12-11T19:33:10.319673+0000","last_clean":"2023-12-11T19:33:10.319673+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:33:10.319673+0000","last_undegraded":"2023-12-11T19:33:10.319673+0000","last_fullsized":"2023-12-11T19:33:10.319673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331108+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:33:10.331108+0000","last_peered":"2023-12-11T19:33:10.331108+0000","last_clean":"2023-12-11T19:33:10.331108+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:33:10.331108+0000","last_undegraded":"2023-12-11T19:33:10.331108+0000","last_fullsized":"2023-12-11T19:33:10.331108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337463+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:33:10.337463+0000","last_peered":"2023-12-11T19:33:10.337463+0000","last_clean":"2023-12-11T19:33:10.337463+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:33:10.337463+0000","last_undegraded":"2023-12-11T19:33:10.337463+0000","last_fullsized":"2023-12-11T19:33:10.337463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343501+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:33:10.343501+0000","last_peered":"2023-12-11T19:33:10.343501+0000","last_clean":"2023-12-11T19:33:10.343501+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:33:10.343501+0000","last_undegraded":"2023-12-11T19:33:10.343501+0000","last_fullsized":"2023-12-11T19:33:10.343501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341666+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:33:10.341666+0000","last_peered":"2023-12-11T19:33:10.341666+0000","last_clean":"2023-12-11T19:33:10.341666+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:33:10.341666+0000","last_undegraded":"2023-12-11T19:33:10.341666+0000","last_fullsized":"2023-12-11T19:33:10.341666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337779+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:33:10.337779+0000","last_peered":"2023-12-11T19:33:10.337779+0000","last_clean":"2023-12-11T19:33:10.337779+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:33:10.337779+0000","last_undegraded":"2023-12-11T19:33:10.337779+0000","last_fullsized":"2023-12-11T19:33:10.337779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70230,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1035,"num_read_kb":603,"num_write":253,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2181,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":210690,"data_compressed":5496,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":460267,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.94399999999999995,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80300000000000005,"5min":1.454,"15min":1.454},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95299999999999996,"5min":1.331,"15min":1.331},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.65800000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.077,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96599999999999997,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93400000000000005,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.60399999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.052,"5min":1.518,"15min":1.518},"last":0.80700000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.85099999999999998,"15min":0.871},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0720000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.85499999999999998,"15min":0.873},"min":{"1min":0.58399999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0980000000000001,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.77900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.115,"5min":1.403,"15min":1.403},"last":1},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.64200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88400000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.86199999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.69899999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1419999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.79200000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.165,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.67500000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0629999999999999,"5min":1.49,"15min":1.49},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.76100000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1819999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.90000000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1112588,"data_compressed":43838,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.8120000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.36899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91000000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.57599999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.96099999999999997,"5min":1.04,"15min":1.04},"last":0.87},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.032,"15min":1.032},"last":0.93700000000000006}]},{"osd":3,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.24},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95299999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.66600000000000004,"15min":0.70199999999999996},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76700000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.81100000000000005},"min":{"1min":0.61699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98399999999999999,"5min":1.085,"15min":1.085},"last":0.90600000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.58999999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97099999999999997,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89800000000000002,"15min":0.90400000000000003},"min":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.89400000000000002,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.626,"15min":0.626},"max":{"1min":0.995,"5min":1.643,"15min":1.643},"last":1.026}]},{"osd":9,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.69499999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0629999999999999,"5min":1.321,"15min":1.321},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.871,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.98899999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.97499999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.873},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1259999999999999,"5min":1.224,"15min":1.224},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90700000000000003,"15min":0.90200000000000002},"min":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1379999999999999,"5min":1.161,"15min":1.161},"last":0.99099999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":459822,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68899999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.101,"5min":6.101,"15min":6.101},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.76700000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.85199999999999998,"15min":0.84399999999999997},"min":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0589999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":6.0510000000000002,"5min":6.0510000000000002,"15min":6.0510000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.82899999999999996,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.024,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.875,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.278},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88,"15min":0.90800000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0429999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90500000000000003,"15min":0.93200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.389,"15min":1.389},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.92200000000000004,"15min":0.96799999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0129999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.94699999999999995,"15min":0.97699999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.081,"5min":1.274,"15min":1.274},"last":0.79900000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.95599999999999996,"15min":1.0029999999999999},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97499999999999998,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.97699999999999998,"15min":1.014},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.0309999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.93899999999999995,"15min":0.99099999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.0089999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.85399999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.61799999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":460077,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.621},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":1.097,"15min":1.097},"last":0.501}]},{"osd":1,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.55700000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1479999999999999,"5min":1.351,"15min":1.351},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.81200000000000006,"15min":0.80900000000000005},"min":{"1min":0.61399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.0740000000000001,"5min":2.21,"15min":2.21},"last":0.88400000000000001}]},{"osd":2,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.292,"15min":1.292},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.2350000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.85599999999999998}]},{"osd":5,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.58199999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1140000000000001,"5min":2.173,"15min":2.173},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.083,"5min":1.177,"15min":1.177},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.84899999999999998,"15min":0.82099999999999995},"min":{"1min":0.74199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2869999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1459999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.71699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.304,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.74099999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.89800000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.86599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.256,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.01,"15min":0.97499999999999998},"min":{"1min":0.94099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3600000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.82299999999999995}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.79200000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.405,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.056,"5min":0.98299999999999998,"15min":0.97099999999999997},"min":{"1min":0.77600000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.343,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.96799999999999997}]},{"osd":11,"last update":"Mon Dec 11 19:32:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.995},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.0369999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1054433,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.51600000000000001,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76900000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59599999999999997},"min":{"1min":0.35399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.165},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77700000000000002,"15min":0.77000000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.89000000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.754,"15min":0.80600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97199999999999998,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70399999999999996,"15min":0.73799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59299999999999997,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73199999999999998,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.042,"5min":1.238,"15min":1.238},"last":0.626}]},{"osd":5,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96899999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.026,"5min":1.28,"15min":1.28},"last":0.998}]},{"osd":6,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79900000000000004,"15min":0.85499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.512,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.751,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.83699999999999997,"5min":1.359,"15min":1.359},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97999999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.85999999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.872}]},{"osd":9,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.98399999999999999,"5min":1.296,"15min":1.296},"last":0.753}]},{"osd":10,"last update":"Mon Dec 11 19:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.90700000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0369999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.194,"5min":1.25,"15min":1.25},"last":0.77700000000000002}]},{"osd":11,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.028},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.94499999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459920,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48299999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85999999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.046,"5min":1.089,"15min":1.089},"last":0.98399999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73099999999999998,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0269999999999999,"5min":1.284,"15min":1.284},"last":0.75},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.494,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.85399999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80000000000000004},"min":{"1min":0.58099999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.0860000000000001,"5min":1.157,"15min":1.157},"last":0.88700000000000001}]},{"osd":5,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.84099999999999997,"15min":0.81799999999999995},"min":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86899999999999999,"15min":0.91600000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.603,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.873}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88500000000000001,"15min":0.87},"min":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.137,"5min":1.234,"15min":1.234},"last":0.94299999999999995}]},{"osd":8,"last update":"Mon Dec 11 19:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.88300000000000001,"15min":0.88500000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.83199999999999996}]},{"osd":9,"last update":"Mon Dec 11 19:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.90900000000000003,"15min":0.91700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3220000000000001,"5min":1.345,"15min":1.345},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.66800000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:33:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83599999999999997,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.94699999999999995,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.96699999999999997,"15min":0.97099999999999997},"min":{"1min":0.70999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.081,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.006}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521484,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.55700000000000005},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69099999999999995,"15min":0.58199999999999996},"min":{"1min":0.58099999999999996,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":1.05,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":14.102}]},{"osd":2,"last update":"Mon Dec 11 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.79900000000000004,"15min":0.72099999999999997},"min":{"1min":0.67600000000000005,"5min":0.42399999999999999,"15min":0.38400000000000001},"max":{"1min":1.1399999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.752,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Dec 11 19:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.435,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.99299999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.81100000000000005,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":14.044,"5min":14.044,"15min":14.044},"last":14.044}]},{"osd":4,"last update":"Mon Dec 11 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80600000000000005,"5min":1.022,"15min":1.022},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.66600000000000004,"15min":0.57199999999999995},"min":{"1min":0.61299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.083,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.86099999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.749,"15min":0.68100000000000005},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1539999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.748},{"interface":"front","average":{"1min":0.745,"5min":0.73599999999999999,"15min":0.85599999999999998},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93600000000000005,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Dec 11 19:32:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.77800000000000002,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":14.17},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.627,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.74199999999999999,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.68000000000000005,"15min":0.627},"min":{"1min":0.501,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69599999999999995}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2110000000000001,"5min":1.52,"15min":1.52},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.091,"5min":0.88500000000000001,"15min":0.83799999999999997},"min":{"1min":0.67100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.82599999999999996}]},{"osd":9,"last update":"Mon Dec 11 19:33:10 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.91700000000000004,"15min":0.875},"min":{"1min":0.77400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.343,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.008,"5min":0.91800000000000004,"15min":0.88400000000000001},"min":{"1min":0.81200000000000006,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.88}]},{"osd":10,"last update":"Mon Dec 11 19:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.83599999999999997,"15min":0.82799999999999996},"min":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0840000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.04,"5min":0.94799999999999995,"15min":0.93300000000000005},"min":{"1min":0.76400000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.196,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.93300000000000005}]},{"osd":11,"last update":"Mon Dec 11 19:32:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.84099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":461253,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82799999999999996,"15min":0.67800000000000005},"min":{"1min":0.52300000000000002,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":1.1890000000000001,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":1.0449999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.748,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0189999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.41399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.403},"last":0.63500000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71899999999999997,"15min":0.67800000000000005},"min":{"1min":0.55100000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.64900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3799999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79400000000000004,"15min":0.81799999999999995},"min":{"1min":0.58599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.4099999999999999,"5min":1.819,"15min":1.819},"last":1.0209999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.081,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.038},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.80700000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.90400000000000003,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1579999999999999,"5min":1.25,"15min":1.25},"last":0.98799999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96399999999999997,"15min":1.0069999999999999},"min":{"1min":0.57099999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92000000000000004,"15min":0.95899999999999996},"min":{"1min":0.58699999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.85199999999999998}]},{"osd":10,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.085},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.88900000000000001}]},{"osd":11,"last update":"Mon Dec 11 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.90100000000000002,"15min":0.88100000000000001},"min":{"1min":0.82099999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1850000000000001,"5min":0.95799999999999996,"15min":0.92000000000000004},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":6.9299999999999997,"5min":6.9299999999999997,"15min":6.9299999999999997},"last":1.103}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":519513,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.753,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.95199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71599999999999997,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":1.0760000000000001,"5min":1.115,"15min":1.115},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.64800000000000002},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Dec 11 19:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63700000000000001,"15min":0.60399999999999998},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.65200000000000002},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Mon Dec 11 19:32:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.93799999999999994}]},{"osd":5,"last update":"Mon Dec 11 19:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.97399999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86499999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Dec 11 19:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.747,"15min":0.70099999999999996},"min":{"1min":0.61799999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76800000000000002,"15min":0.747},"min":{"1min":0.64000000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0660000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.751},"min":{"1min":0.59499999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.119,"5min":1.133,"15min":1.133},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.86099999999999999,"15min":0.83299999999999996},"min":{"1min":0.70299999999999996,"5min":0.627,"15min":0.627},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.86599999999999999},"min":{"1min":0.68000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.137,"5min":1.167,"15min":1.167},"last":0.92100000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.91400000000000003,"15min":0.89500000000000002},"min":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.095,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91500000000000004,"15min":0.88900000000000001},"min":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.109,"5min":1.169,"15min":1.169},"last":0.97799999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85399999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":458799,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48499999999999999,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":0.80700000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.28599999999999998},"max":{"1min":0.93400000000000005,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.625},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98999999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.93999999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.64700000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.96999999999999997,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.65100000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.36199999999999999},"max":{"1min":1.0660000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88}]},{"osd":4,"last update":"Mon Dec 11 19:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.751,"15min":0.66700000000000004},"min":{"1min":0.57599999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.80500000000000005,"15min":0.71699999999999997},"min":{"1min":0.65300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.91000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.198,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77000000000000002,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1220000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.006}]},{"osd":6,"last update":"Mon Dec 11 19:32:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.196,"5min":1.28,"15min":1.28},"last":0.746}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86499999999999999,"15min":0.82699999999999996},"min":{"1min":0.68600000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1399999999999999,"5min":1.752,"15min":1.752},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86499999999999999,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.206,"5min":1.631,"15min":1.631},"last":1.0620000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93899999999999995,"15min":0.96399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.042},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.95899999999999996,"15min":1.0029999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3220000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.95399999999999996,"15min":0.97299999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.98499999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.111},{"interface":"front","average":{"1min":0.88700000000000001,"5min":1.0189999999999999,"15min":1.0549999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1000000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.095}]},{"osd":11,"last update":"Mon Dec 11 19:32:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.141},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0269999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461955,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.496},"min":{"1min":0.39400000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85199999999999998,"5min":1.294,"15min":1.294},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90500000000000003,"5min":1.018,"15min":1.018},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.58899999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0209999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.98699999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69999999999999996,"15min":0.65900000000000003},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0680000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.877},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.76900000000000002,"15min":0.70799999999999996},"min":{"1min":0.57799999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0620000000000001,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.71999999999999997},"min":{"1min":0.51800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Mon Dec 11 19:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72999999999999998,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.395,"5min":1.429,"15min":1.429},"last":0.94199999999999995}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82599999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.997,"5min":1.26,"15min":1.26},"last":0.96199999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.67600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0369999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.109,"5min":1.494,"15min":1.494},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.81799999999999995,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1100000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94399999999999995,"15min":0.95499999999999996},"min":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0960000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Mon Dec 11 19:32:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.999}]}]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1052916,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.624},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83799999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.65200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83899999999999997,"5min":1.177,"15min":1.177},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.53000000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71999999999999997,"15min":0.69199999999999995},"min":{"1min":0.56299999999999994,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98999999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.47099999999999997},"min":{"1min":0.372,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.879,"15min":0.879},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94499999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52400000000000002}]},{"osd":3,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.74199999999999999},"min":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96699999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.83399999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96499999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91000000000000003,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.58799999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.745},"min":{"1min":0.55500000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1240000000000001,"5min":1.597,"15min":1.597},"last":0.66500000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78000000000000003,"15min":0.75},"min":{"1min":0.58799999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.751},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.83999999999999997,"15min":0.80900000000000005},"min":{"1min":0.71599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0760000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.214,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.873},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77200000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83899999999999997},"min":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.88100000000000001,"15min":0.88800000000000001},"min":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2509999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Mon Dec 11 19:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91500000000000004,"15min":0.90900000000000003},"min":{"1min":0.81399999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1060000000000001,"5min":1.548,"15min":1.548},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.002,"5min":0.94499999999999995,"15min":0.93100000000000005},"min":{"1min":0.77200000000000002,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64674,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60936,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63520,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:33:42.038 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T19:33:42.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:41 smithi070 ceph-mon[104044]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-12-11T19:33:42.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:41 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1646709396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:42.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:41 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4140420137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:42.133 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:41 smithi066 ceph-mon[97920]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-12-11T19:33:42.133 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:41 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1646709396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:42.133 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:41 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4140420137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:42.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:41 smithi167 ceph-mon[102750]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-12-11T19:33:42.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:41 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1646709396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:42.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:41 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4140420137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 -- 172.21.15.66:0/827019665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec104830 msgr2=0x7ff2ec106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/827019665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec104830 0x7ff2ec106c20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009990 tx=0x7ff2dc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 -- 172.21.15.66:0/827019665 shutdown_connections 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/827019665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2ec107160 0x7ff2ec1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/827019665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ec101f00 0x7ff2ec1042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/827019665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec104830 0x7ff2ec106c20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 -- 172.21.15.66:0/827019665 >> 172.21.15.66:0/827019665 conn(0x7ff2ec0fb8f0 msgr2=0x7ff2ec0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 -- 172.21.15.66:0/827019665 shutdown_connections 2023-12-11T19:33:42.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.278+0000 7ff2f35ff700 1 -- 172.21.15.66:0/827019665 wait complete. 2023-12-11T19:33:42.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.279+0000 7ff2f35ff700 1 Processor -- start 2023-12-11T19:33:42.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.279+0000 7ff2f35ff700 1 -- start start 2023-12-11T19:33:42.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f35ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ec101f00 0x7ff2ec199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f35ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2ec104830 0x7ff2ec199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f35ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 0x7ff2ec1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f35ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec1a1080 con 0x7ff2ec107160 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f35ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec1a11c0 con 0x7ff2ec104830 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f35ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec1a1350 con 0x7ff2ec101f00 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f1b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 0x7ff2ec1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f1b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 0x7ff2ec1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50780/0 (socket says 172.21.15.66:50780) 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f1b9c700 1 -- 172.21.15.66:0/2361960597 learned_addr learned my addr 172.21.15.66:0/2361960597 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f139b700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ec101f00 0x7ff2ec199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f0b9a700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2ec104830 0x7ff2ec199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f1b9c700 1 -- 172.21.15.66:0/2361960597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ec101f00 msgr2=0x7ff2ec199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.280+0000 7ff2f1b9c700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ec101f00 0x7ff2ec199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2f1b9c700 1 -- 172.21.15.66:0/2361960597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2ec104830 msgr2=0x7ff2ec199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2f1b9c700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2ec104830 0x7ff2ec199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2f1b9c700 1 -- 172.21.15.66:0/2361960597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0095f0 con 0x7ff2ec107160 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2f1b9c700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 0x7ff2ec1a0ab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400ef30 tx=0x7ff2e400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4018070 con 0x7ff2ec107160 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2e400f040 con 0x7ff2ec107160 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec1a15d0 con 0x7ff2ec107160 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4003d00 con 0x7ff2ec107160 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.281+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec1a1ae0 con 0x7ff2ec107160 2023-12-11T19:33:42.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.282+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec108d60 con 0x7ff2ec107160 2023-12-11T19:33:42.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.284+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff2e400caa0 con 0x7ff2ec107160 2023-12-11T19:33:42.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.285+0000 7ff2e27fc700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff2d80a1500 0x7ff2d80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.286+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2e40cca10 con 0x7ff2ec107160 2023-12-11T19:33:42.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.286+0000 7ff2f139b700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff2d80a1500 0x7ff2d80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.286+0000 7ff2f139b700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff2d80a1500 0x7ff2d80a39c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009bf0 tx=0x7ff2e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:42.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.289+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff2e40960c0 con 0x7ff2ec107160 2023-12-11T19:33:42.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.499+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2ec02d3d0 con 0x7ff2d80a1500 2023-12-11T19:33:42.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.510+0000 7ff2e27fc700 1 -- 172.21.15.66:0/2361960597 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343487 (secure 0 0 0) 0x7ff2ec02d3d0 con 0x7ff2d80a1500 2023-12-11T19:33:42.515 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff2d80a1500 msgr2=0x7ff2d80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff2d80a1500 0x7ff2d80a39c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009bf0 tx=0x7ff2e8009260 comp rx=0 tx=0).stop 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 msgr2=0x7ff2ec1a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 0x7ff2ec1a0ab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400ef30 tx=0x7ff2e400c450 comp rx=0 tx=0).stop 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 shutdown_connections 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff2d80a1500 0x7ff2d80a39c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2ec107160 0x7ff2ec1a0ab0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ec101f00 0x7ff2ec199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 --2- 172.21.15.66:0/2361960597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2ec104830 0x7ff2ec199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 >> 172.21.15.66:0/2361960597 conn(0x7ff2ec0fb8f0 msgr2=0x7ff2ec1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:42.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 shutdown_connections 2023-12-11T19:33:42.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.515+0000 7ff2f35ff700 1 -- 172.21.15.66:0/2361960597 wait complete. 2023-12-11T19:33:42.519 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:33:42.535 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2023-12-11T19:33:42.312135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660844,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1171,"num_read_kb":717,"num_write":438,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1183040,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335976,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605848,"statfs":{"total":1151940427776,"available":1151596388352,"internally_reserved":0,"allocated":12066816,"data_stored":7484521,"data_compressed":183741,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2490,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002955"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343455+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:33:10.343455+0000","last_peered":"2023-12-11T19:33:10.343455+0000","last_clean":"2023-12-11T19:33:10.343455+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:33:10.343455+0000","last_undegraded":"2023-12-11T19:33:10.343455+0000","last_fullsized":"2023-12-11T19:33:10.343455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.334250+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:33:10.334250+0000","last_peered":"2023-12-11T19:33:10.334250+0000","last_clean":"2023-12-11T19:33:10.334250+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:33:10.334250+0000","last_undegraded":"2023-12-11T19:33:10.334250+0000","last_fullsized":"2023-12-11T19:33:10.334250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340675+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:33:10.340675+0000","last_peered":"2023-12-11T19:33:10.340675+0000","last_clean":"2023-12-11T19:33:10.340675+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:33:10.340675+0000","last_undegraded":"2023-12-11T19:33:10.340675+0000","last_fullsized":"2023-12-11T19:33:10.340675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.336763+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:33:10.336763+0000","last_peered":"2023-12-11T19:33:10.336763+0000","last_clean":"2023-12-11T19:33:10.336763+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:33:10.336763+0000","last_undegraded":"2023-12-11T19:33:10.336763+0000","last_fullsized":"2023-12-11T19:33:10.336763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324900+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:33:10.324900+0000","last_peered":"2023-12-11T19:33:10.324900+0000","last_clean":"2023-12-11T19:33:10.324900+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:33:10.324900+0000","last_undegraded":"2023-12-11T19:33:10.324900+0000","last_fullsized":"2023-12-11T19:33:10.324900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324677+0000","last_change":"2023-12-11T19:32:11.585539+0000","last_active":"2023-12-11T19:33:10.324677+0000","last_peered":"2023-12-11T19:33:10.324677+0000","last_clean":"2023-12-11T19:33:10.324677+0000","last_became_active":"2023-12-11T19:32:11.585087+0000","last_became_peered":"2023-12-11T19:32:11.585087+0000","last_unstale":"2023-12-11T19:33:10.324677+0000","last_undegraded":"2023-12-11T19:33:10.324677+0000","last_fullsized":"2023-12-11T19:33:10.324677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:23.076166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332857+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:33:10.332857+0000","last_peered":"2023-12-11T19:33:10.332857+0000","last_clean":"2023-12-11T19:33:10.332857+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:33:10.332857+0000","last_undegraded":"2023-12-11T19:33:10.332857+0000","last_fullsized":"2023-12-11T19:33:10.332857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341918+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:33:10.341918+0000","last_peered":"2023-12-11T19:33:10.341918+0000","last_clean":"2023-12-11T19:33:10.341918+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:33:10.341918+0000","last_undegraded":"2023-12-11T19:33:10.341918+0000","last_fullsized":"2023-12-11T19:33:10.341918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343323+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:33:10.343323+0000","last_peered":"2023-12-11T19:33:10.343323+0000","last_clean":"2023-12-11T19:33:10.343323+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:33:10.343323+0000","last_undegraded":"2023-12-11T19:33:10.343323+0000","last_fullsized":"2023-12-11T19:33:10.343323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339811+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:33:10.339811+0000","last_peered":"2023-12-11T19:33:10.339811+0000","last_clean":"2023-12-11T19:33:10.339811+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:33:10.339811+0000","last_undegraded":"2023-12-11T19:33:10.339811+0000","last_fullsized":"2023-12-11T19:33:10.339811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339748+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:33:10.339748+0000","last_peered":"2023-12-11T19:33:10.339748+0000","last_clean":"2023-12-11T19:33:10.339748+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:33:10.339748+0000","last_undegraded":"2023-12-11T19:33:10.339748+0000","last_fullsized":"2023-12-11T19:33:10.339748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331280+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:33:10.331280+0000","last_peered":"2023-12-11T19:33:10.331280+0000","last_clean":"2023-12-11T19:33:10.331280+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:33:10.331280+0000","last_undegraded":"2023-12-11T19:33:10.331280+0000","last_fullsized":"2023-12-11T19:33:10.331280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337449+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:33:10.337449+0000","last_peered":"2023-12-11T19:33:10.337449+0000","last_clean":"2023-12-11T19:33:10.337449+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:33:10.337449+0000","last_undegraded":"2023-12-11T19:33:10.337449+0000","last_fullsized":"2023-12-11T19:33:10.337449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.334371+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:33:10.334371+0000","last_peered":"2023-12-11T19:33:10.334371+0000","last_clean":"2023-12-11T19:33:10.334371+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:33:10.334371+0000","last_undegraded":"2023-12-11T19:33:10.334371+0000","last_fullsized":"2023-12-11T19:33:10.334371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341068+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:33:10.341068+0000","last_peered":"2023-12-11T19:33:10.341068+0000","last_clean":"2023-12-11T19:33:10.341068+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:33:10.341068+0000","last_undegraded":"2023-12-11T19:33:10.341068+0000","last_fullsized":"2023-12-11T19:33:10.341068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344589+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:33:10.344589+0000","last_peered":"2023-12-11T19:33:10.344589+0000","last_clean":"2023-12-11T19:33:10.344589+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:33:10.344589+0000","last_undegraded":"2023-12-11T19:33:10.344589+0000","last_fullsized":"2023-12-11T19:33:10.344589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343527+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:33:10.343527+0000","last_peered":"2023-12-11T19:33:10.343527+0000","last_clean":"2023-12-11T19:33:10.343527+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:33:10.343527+0000","last_undegraded":"2023-12-11T19:33:10.343527+0000","last_fullsized":"2023-12-11T19:33:10.343527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333632+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:33:10.333632+0000","last_peered":"2023-12-11T19:33:10.333632+0000","last_clean":"2023-12-11T19:33:10.333632+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:33:10.333632+0000","last_undegraded":"2023-12-11T19:33:10.333632+0000","last_fullsized":"2023-12-11T19:33:10.333632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324560+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:33:10.324560+0000","last_peered":"2023-12-11T19:33:10.324560+0000","last_clean":"2023-12-11T19:33:10.324560+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:33:10.324560+0000","last_undegraded":"2023-12-11T19:33:10.324560+0000","last_fullsized":"2023-12-11T19:33:10.324560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:37.707150+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:33:37.707150+0000","last_peered":"2023-12-11T19:33:37.707150+0000","last_clean":"2023-12-11T19:33:37.707150+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:33:37.707150+0000","last_undegraded":"2023-12-11T19:33:37.707150+0000","last_fullsized":"2023-12-11T19:33:37.707150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.328995+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:33:10.328995+0000","last_peered":"2023-12-11T19:33:10.328995+0000","last_clean":"2023-12-11T19:33:10.328995+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:33:10.328995+0000","last_undegraded":"2023-12-11T19:33:10.328995+0000","last_fullsized":"2023-12-11T19:33:10.328995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586855+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:32:10.586855+0000","last_peered":"2023-12-11T19:32:10.586855+0000","last_clean":"2023-12-11T19:32:10.586855+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:32:10.586855+0000","last_undegraded":"2023-12-11T19:32:10.586855+0000","last_fullsized":"2023-12-11T19:32:10.586855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344983+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:33:10.344983+0000","last_peered":"2023-12-11T19:33:10.344983+0000","last_clean":"2023-12-11T19:33:10.344983+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:33:10.344983+0000","last_undegraded":"2023-12-11T19:33:10.344983+0000","last_fullsized":"2023-12-11T19:33:10.344983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585009+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:32:10.585009+0000","last_peered":"2023-12-11T19:32:10.585009+0000","last_clean":"2023-12-11T19:32:10.585009+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:32:10.585009+0000","last_undegraded":"2023-12-11T19:32:10.585009+0000","last_fullsized":"2023-12-11T19:32:10.585009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343252+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:33:10.343252+0000","last_peered":"2023-12-11T19:33:10.343252+0000","last_clean":"2023-12-11T19:33:10.343252+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:33:10.343252+0000","last_undegraded":"2023-12-11T19:33:10.343252+0000","last_fullsized":"2023-12-11T19:33:10.343252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584594+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:32:10.584594+0000","last_peered":"2023-12-11T19:32:10.584594+0000","last_clean":"2023-12-11T19:32:10.584594+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:32:10.584594+0000","last_undegraded":"2023-12-11T19:32:10.584594+0000","last_fullsized":"2023-12-11T19:32:10.584594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339985+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:33:10.339985+0000","last_peered":"2023-12-11T19:33:10.339985+0000","last_clean":"2023-12-11T19:33:10.339985+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:33:10.339985+0000","last_undegraded":"2023-12-11T19:33:10.339985+0000","last_fullsized":"2023-12-11T19:33:10.339985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:37.733788+0000","last_change":"2023-12-11T19:32:11.601552+0000","last_active":"2023-12-11T19:33:37.733788+0000","last_peered":"2023-12-11T19:33:37.733788+0000","last_clean":"2023-12-11T19:33:37.733788+0000","last_became_active":"2023-12-11T19:32:11.593828+0000","last_became_peered":"2023-12-11T19:32:11.593828+0000","last_unstale":"2023-12-11T19:33:37.733788+0000","last_undegraded":"2023-12-11T19:33:37.733788+0000","last_fullsized":"2023-12-11T19:33:37.733788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:53.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1650,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333342+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:33:10.333342+0000","last_peered":"2023-12-11T19:33:10.333342+0000","last_clean":"2023-12-11T19:33:10.333342+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:33:10.333342+0000","last_undegraded":"2023-12-11T19:33:10.333342+0000","last_fullsized":"2023-12-11T19:33:10.333342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586092+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:32:10.586092+0000","last_peered":"2023-12-11T19:32:10.586092+0000","last_clean":"2023-12-11T19:32:10.586092+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:32:10.586092+0000","last_undegraded":"2023-12-11T19:32:10.586092+0000","last_fullsized":"2023-12-11T19:32:10.586092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.338447+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:33:10.338447+0000","last_peered":"2023-12-11T19:33:10.338447+0000","last_clean":"2023-12-11T19:33:10.338447+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:33:10.338447+0000","last_undegraded":"2023-12-11T19:33:10.338447+0000","last_fullsized":"2023-12-11T19:33:10.338447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581195+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:32:10.581195+0000","last_peered":"2023-12-11T19:32:10.581195+0000","last_clean":"2023-12-11T19:32:10.581195+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:32:10.581195+0000","last_undegraded":"2023-12-11T19:32:10.581195+0000","last_fullsized":"2023-12-11T19:32:10.581195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337715+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:33:10.337715+0000","last_peered":"2023-12-11T19:33:10.337715+0000","last_clean":"2023-12-11T19:33:10.337715+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:33:10.337715+0000","last_undegraded":"2023-12-11T19:33:10.337715+0000","last_fullsized":"2023-12-11T19:33:10.337715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581346+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:32:10.581346+0000","last_peered":"2023-12-11T19:32:10.581346+0000","last_clean":"2023-12-11T19:32:10.581346+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:32:10.581346+0000","last_undegraded":"2023-12-11T19:32:10.581346+0000","last_fullsized":"2023-12-11T19:32:10.581346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342744+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:33:10.342744+0000","last_peered":"2023-12-11T19:33:10.342744+0000","last_clean":"2023-12-11T19:33:10.342744+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:33:10.342744+0000","last_undegraded":"2023-12-11T19:33:10.342744+0000","last_fullsized":"2023-12-11T19:33:10.342744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":227,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.732269+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:33:39.732269+0000","last_peered":"2023-12-11T19:33:39.732269+0000","last_clean":"2023-12-11T19:33:39.732269+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:33:39.732269+0000","last_undegraded":"2023-12-11T19:33:39.732269+0000","last_fullsized":"2023-12-11T19:33:39.732269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340936+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:33:10.340936+0000","last_peered":"2023-12-11T19:33:10.340936+0000","last_clean":"2023-12-11T19:33:10.340936+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:33:10.340936+0000","last_undegraded":"2023-12-11T19:33:10.340936+0000","last_fullsized":"2023-12-11T19:33:10.340936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578707+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:32:10.578707+0000","last_peered":"2023-12-11T19:32:10.578707+0000","last_clean":"2023-12-11T19:32:10.578707+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:32:10.578707+0000","last_undegraded":"2023-12-11T19:32:10.578707+0000","last_fullsized":"2023-12-11T19:32:10.578707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340428+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:33:10.340428+0000","last_peered":"2023-12-11T19:33:10.340428+0000","last_clean":"2023-12-11T19:33:10.340428+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:33:10.340428+0000","last_undegraded":"2023-12-11T19:33:10.340428+0000","last_fullsized":"2023-12-11T19:33:10.340428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581870+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:32:10.581870+0000","last_peered":"2023-12-11T19:32:10.581870+0000","last_clean":"2023-12-11T19:32:10.581870+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:32:10.581870+0000","last_undegraded":"2023-12-11T19:32:10.581870+0000","last_fullsized":"2023-12-11T19:32:10.581870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344002+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:33:10.344002+0000","last_peered":"2023-12-11T19:33:10.344002+0000","last_clean":"2023-12-11T19:33:10.344002+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:33:10.344002+0000","last_undegraded":"2023-12-11T19:33:10.344002+0000","last_fullsized":"2023-12-11T19:33:10.344002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":267,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.658525+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:33:39.658525+0000","last_peered":"2023-12-11T19:33:39.658525+0000","last_clean":"2023-12-11T19:33:39.658525+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:33:39.658525+0000","last_undegraded":"2023-12-11T19:33:39.658525+0000","last_fullsized":"2023-12-11T19:33:39.658525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60756,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":168,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333830+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:33:10.333830+0000","last_peered":"2023-12-11T19:33:10.333830+0000","last_clean":"2023-12-11T19:33:10.333830+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:33:10.333830+0000","last_undegraded":"2023-12-11T19:33:10.333830+0000","last_fullsized":"2023-12-11T19:33:10.333830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.587382+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:32:10.587382+0000","last_peered":"2023-12-11T19:32:10.587382+0000","last_clean":"2023-12-11T19:32:10.587382+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:32:10.587382+0000","last_undegraded":"2023-12-11T19:32:10.587382+0000","last_fullsized":"2023-12-11T19:32:10.587382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337785+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:33:10.337785+0000","last_peered":"2023-12-11T19:33:10.337785+0000","last_clean":"2023-12-11T19:33:10.337785+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:33:10.337785+0000","last_undegraded":"2023-12-11T19:33:10.337785+0000","last_fullsized":"2023-12-11T19:33:10.337785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581282+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:32:10.581282+0000","last_peered":"2023-12-11T19:32:10.581282+0000","last_clean":"2023-12-11T19:32:10.581282+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:32:10.581282+0000","last_undegraded":"2023-12-11T19:32:10.581282+0000","last_fullsized":"2023-12-11T19:32:10.581282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344461+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:33:10.344461+0000","last_peered":"2023-12-11T19:33:10.344461+0000","last_clean":"2023-12-11T19:33:10.344461+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:33:10.344461+0000","last_undegraded":"2023-12-11T19:33:10.344461+0000","last_fullsized":"2023-12-11T19:33:10.344461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'8","reported_seq":126,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:36.732277+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:33:36.732277+0000","last_peered":"2023-12-11T19:33:36.732277+0000","last_clean":"2023-12-11T19:33:36.732277+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:33:36.732277+0000","last_undegraded":"2023-12-11T19:33:36.732277+0000","last_fullsized":"2023-12-11T19:33:36.732277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1616,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341155+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:33:10.341155+0000","last_peered":"2023-12-11T19:33:10.341155+0000","last_clean":"2023-12-11T19:33:10.341155+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:33:10.341155+0000","last_undegraded":"2023-12-11T19:33:10.341155+0000","last_fullsized":"2023-12-11T19:33:10.341155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583186+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:32:10.583186+0000","last_peered":"2023-12-11T19:32:10.583186+0000","last_clean":"2023-12-11T19:32:10.583186+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:32:10.583186+0000","last_undegraded":"2023-12-11T19:32:10.583186+0000","last_fullsized":"2023-12-11T19:32:10.583186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339499+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:33:10.339499+0000","last_peered":"2023-12-11T19:33:10.339499+0000","last_clean":"2023-12-11T19:33:10.339499+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:33:10.339499+0000","last_undegraded":"2023-12-11T19:33:10.339499+0000","last_fullsized":"2023-12-11T19:33:10.339499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.583539+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:32:11.583539+0000","last_peered":"2023-12-11T19:32:11.583539+0000","last_clean":"2023-12-11T19:32:11.583539+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:32:11.583539+0000","last_undegraded":"2023-12-11T19:32:11.583539+0000","last_fullsized":"2023-12-11T19:32:11.583539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332391+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:33:10.332391+0000","last_peered":"2023-12-11T19:33:10.332391+0000","last_clean":"2023-12-11T19:33:10.332391+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:33:10.332391+0000","last_undegraded":"2023-12-11T19:33:10.332391+0000","last_fullsized":"2023-12-11T19:33:10.332391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583392+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:32:10.583392+0000","last_peered":"2023-12-11T19:32:10.583392+0000","last_clean":"2023-12-11T19:32:10.583392+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:32:10.583392+0000","last_undegraded":"2023-12-11T19:32:10.583392+0000","last_fullsized":"2023-12-11T19:32:10.583392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324798+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:33:10.324798+0000","last_peered":"2023-12-11T19:33:10.324798+0000","last_clean":"2023-12-11T19:33:10.324798+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:33:10.324798+0000","last_undegraded":"2023-12-11T19:33:10.324798+0000","last_fullsized":"2023-12-11T19:33:10.324798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:36.706343+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:33:36.706343+0000","last_peered":"2023-12-11T19:33:36.706343+0000","last_clean":"2023-12-11T19:33:36.706343+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:33:36.706343+0000","last_undegraded":"2023-12-11T19:33:36.706343+0000","last_fullsized":"2023-12-11T19:33:36.706343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1086,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341698+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:33:10.341698+0000","last_peered":"2023-12-11T19:33:10.341698+0000","last_clean":"2023-12-11T19:33:10.341698+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:33:10.341698+0000","last_undegraded":"2023-12-11T19:33:10.341698+0000","last_fullsized":"2023-12-11T19:33:10.341698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.317738+0000","last_change":"2023-12-11T19:32:11.594139+0000","last_active":"2023-12-11T19:33:10.317738+0000","last_peered":"2023-12-11T19:33:10.317738+0000","last_clean":"2023-12-11T19:33:10.317738+0000","last_became_active":"2023-12-11T19:32:11.593876+0000","last_became_peered":"2023-12-11T19:32:11.593876+0000","last_unstale":"2023-12-11T19:33:10.317738+0000","last_undegraded":"2023-12-11T19:33:10.317738+0000","last_fullsized":"2023-12-11T19:33:10.317738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:22:15.243111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":125,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.736520+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:33:39.736520+0000","last_peered":"2023-12-11T19:33:39.736520+0000","last_clean":"2023-12-11T19:33:39.736520+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:33:39.736520+0000","last_undegraded":"2023-12-11T19:33:39.736520+0000","last_fullsized":"2023-12-11T19:33:39.736520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341374+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:33:10.341374+0000","last_peered":"2023-12-11T19:33:10.341374+0000","last_clean":"2023-12-11T19:33:10.341374+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:33:10.341374+0000","last_undegraded":"2023-12-11T19:33:10.341374+0000","last_fullsized":"2023-12-11T19:33:10.341374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582019+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:32:10.582019+0000","last_peered":"2023-12-11T19:32:10.582019+0000","last_clean":"2023-12-11T19:32:10.582019+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:32:10.582019+0000","last_undegraded":"2023-12-11T19:32:10.582019+0000","last_fullsized":"2023-12-11T19:32:10.582019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":315,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.582905+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:32:11.582905+0000","last_peered":"2023-12-11T19:32:11.582905+0000","last_clean":"2023-12-11T19:32:11.582905+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:32:11.582905+0000","last_undegraded":"2023-12-11T19:32:11.582905+0000","last_fullsized":"2023-12-11T19:32:11.582905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.325546+0000","last_change":"2023-12-11T19:32:11.590972+0000","last_active":"2023-12-11T19:33:10.325546+0000","last_peered":"2023-12-11T19:33:10.325546+0000","last_clean":"2023-12-11T19:33:10.325546+0000","last_became_active":"2023-12-11T19:32:11.590564+0000","last_became_peered":"2023-12-11T19:32:11.590564+0000","last_unstale":"2023-12-11T19:33:10.325546+0000","last_undegraded":"2023-12-11T19:33:10.325546+0000","last_fullsized":"2023-12-11T19:33:10.325546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:48:35.323725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578698+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:32:10.578698+0000","last_peered":"2023-12-11T19:32:10.578698+0000","last_clean":"2023-12-11T19:32:10.578698+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:32:10.578698+0000","last_undegraded":"2023-12-11T19:32:10.578698+0000","last_fullsized":"2023-12-11T19:32:10.578698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584892+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:32:10.584892+0000","last_peered":"2023-12-11T19:32:10.584892+0000","last_clean":"2023-12-11T19:32:10.584892+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:32:10.584892+0000","last_undegraded":"2023-12-11T19:32:10.584892+0000","last_fullsized":"2023-12-11T19:32:10.584892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339768+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:33:10.339768+0000","last_peered":"2023-12-11T19:33:10.339768+0000","last_clean":"2023-12-11T19:33:10.339768+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:33:10.339768+0000","last_undegraded":"2023-12-11T19:33:10.339768+0000","last_fullsized":"2023-12-11T19:33:10.339768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.878510+0000","last_change":"2023-12-11T19:32:11.599025+0000","last_active":"2023-12-11T19:33:10.878510+0000","last_peered":"2023-12-11T19:33:10.878510+0000","last_clean":"2023-12-11T19:33:10.878510+0000","last_became_active":"2023-12-11T19:32:11.587530+0000","last_became_peered":"2023-12-11T19:32:11.587530+0000","last_unstale":"2023-12-11T19:33:10.878510+0000","last_undegraded":"2023-12-11T19:33:10.878510+0000","last_fullsized":"2023-12-11T19:33:10.878510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:08:05.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339794+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:33:10.339794+0000","last_peered":"2023-12-11T19:33:10.339794+0000","last_clean":"2023-12-11T19:33:10.339794+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:33:10.339794+0000","last_undegraded":"2023-12-11T19:33:10.339794+0000","last_fullsized":"2023-12-11T19:33:10.339794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581975+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:32:10.581975+0000","last_peered":"2023-12-11T19:32:10.581975+0000","last_clean":"2023-12-11T19:32:10.581975+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:32:10.581975+0000","last_undegraded":"2023-12-11T19:32:10.581975+0000","last_fullsized":"2023-12-11T19:32:10.581975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342950+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:33:10.342950+0000","last_peered":"2023-12-11T19:33:10.342950+0000","last_clean":"2023-12-11T19:33:10.342950+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:33:10.342950+0000","last_undegraded":"2023-12-11T19:33:10.342950+0000","last_fullsized":"2023-12-11T19:33:10.342950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581192+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:32:10.581192+0000","last_peered":"2023-12-11T19:32:10.581192+0000","last_clean":"2023-12-11T19:32:10.581192+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:32:10.581192+0000","last_undegraded":"2023-12-11T19:32:10.581192+0000","last_fullsized":"2023-12-11T19:32:10.581192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.287117+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:33:10.287117+0000","last_peered":"2023-12-11T19:33:10.287117+0000","last_clean":"2023-12-11T19:33:10.287117+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:33:10.287117+0000","last_undegraded":"2023-12-11T19:33:10.287117+0000","last_fullsized":"2023-12-11T19:33:10.287117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585191+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:32:10.585191+0000","last_peered":"2023-12-11T19:32:10.585191+0000","last_clean":"2023-12-11T19:32:10.585191+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:32:10.585191+0000","last_undegraded":"2023-12-11T19:32:10.585191+0000","last_fullsized":"2023-12-11T19:32:10.585191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339172+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:33:10.339172+0000","last_peered":"2023-12-11T19:33:10.339172+0000","last_clean":"2023-12-11T19:33:10.339172+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:33:10.339172+0000","last_undegraded":"2023-12-11T19:33:10.339172+0000","last_fullsized":"2023-12-11T19:33:10.339172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.588890+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:32:11.588890+0000","last_peered":"2023-12-11T19:32:11.588890+0000","last_clean":"2023-12-11T19:32:11.588890+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:32:11.588890+0000","last_undegraded":"2023-12-11T19:32:11.588890+0000","last_fullsized":"2023-12-11T19:32:11.588890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342653+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:33:10.342653+0000","last_peered":"2023-12-11T19:33:10.342653+0000","last_clean":"2023-12-11T19:33:10.342653+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:33:10.342653+0000","last_undegraded":"2023-12-11T19:33:10.342653+0000","last_fullsized":"2023-12-11T19:33:10.342653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599949+0000","last_change":"2023-12-11T19:32:11.599116+0000","last_active":"2023-12-11T19:32:11.599949+0000","last_peered":"2023-12-11T19:32:11.599949+0000","last_clean":"2023-12-11T19:32:11.599949+0000","last_became_active":"2023-12-11T19:32:11.592106+0000","last_became_peered":"2023-12-11T19:32:11.592106+0000","last_unstale":"2023-12-11T19:32:11.599949+0000","last_undegraded":"2023-12-11T19:32:11.599949+0000","last_fullsized":"2023-12-11T19:32:11.599949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:23.285224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344074+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:33:10.344074+0000","last_peered":"2023-12-11T19:33:10.344074+0000","last_clean":"2023-12-11T19:33:10.344074+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:33:10.344074+0000","last_undegraded":"2023-12-11T19:33:10.344074+0000","last_fullsized":"2023-12-11T19:33:10.344074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.732938+0000","last_change":"2023-12-11T19:32:11.603249+0000","last_active":"2023-12-11T19:33:39.732938+0000","last_peered":"2023-12-11T19:33:39.732938+0000","last_clean":"2023-12-11T19:33:39.732938+0000","last_became_active":"2023-12-11T19:32:11.587129+0000","last_became_peered":"2023-12-11T19:32:11.587129+0000","last_unstale":"2023-12-11T19:33:39.732938+0000","last_undegraded":"2023-12-11T19:33:39.732938+0000","last_fullsized":"2023-12-11T19:33:39.732938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:28:44.762360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1668,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343509+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:33:10.343509+0000","last_peered":"2023-12-11T19:33:10.343509+0000","last_clean":"2023-12-11T19:33:10.343509+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:33:10.343509+0000","last_undegraded":"2023-12-11T19:33:10.343509+0000","last_fullsized":"2023-12-11T19:33:10.343509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581433+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:32:10.581433+0000","last_peered":"2023-12-11T19:32:10.581433+0000","last_clean":"2023-12-11T19:32:10.581433+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:32:10.581433+0000","last_undegraded":"2023-12-11T19:32:10.581433+0000","last_fullsized":"2023-12-11T19:32:10.581433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333570+0000","last_change":"2023-12-11T19:32:11.596576+0000","last_active":"2023-12-11T19:33:10.333570+0000","last_peered":"2023-12-11T19:33:10.333570+0000","last_clean":"2023-12-11T19:33:10.333570+0000","last_became_active":"2023-12-11T19:32:11.592679+0000","last_became_peered":"2023-12-11T19:32:11.592679+0000","last_unstale":"2023-12-11T19:33:10.333570+0000","last_undegraded":"2023-12-11T19:33:10.333570+0000","last_fullsized":"2023-12-11T19:33:10.333570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:26:20.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581645+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:32:10.581645+0000","last_peered":"2023-12-11T19:32:10.581645+0000","last_clean":"2023-12-11T19:32:10.581645+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:32:10.581645+0000","last_undegraded":"2023-12-11T19:32:10.581645+0000","last_fullsized":"2023-12-11T19:32:10.581645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331611+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:33:10.331611+0000","last_peered":"2023-12-11T19:33:10.331611+0000","last_clean":"2023-12-11T19:33:10.331611+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:33:10.331611+0000","last_undegraded":"2023-12-11T19:33:10.331611+0000","last_fullsized":"2023-12-11T19:33:10.331611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.335239+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:33:10.335239+0000","last_peered":"2023-12-11T19:33:10.335239+0000","last_clean":"2023-12-11T19:33:10.335239+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:33:10.335239+0000","last_undegraded":"2023-12-11T19:33:10.335239+0000","last_fullsized":"2023-12-11T19:33:10.335239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333471+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:33:10.333471+0000","last_peered":"2023-12-11T19:33:10.333471+0000","last_clean":"2023-12-11T19:33:10.333471+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:33:10.333471+0000","last_undegraded":"2023-12-11T19:33:10.333471+0000","last_fullsized":"2023-12-11T19:33:10.333471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332291+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:33:10.332291+0000","last_peered":"2023-12-11T19:33:10.332291+0000","last_clean":"2023-12-11T19:33:10.332291+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:33:10.332291+0000","last_undegraded":"2023-12-11T19:33:10.332291+0000","last_fullsized":"2023-12-11T19:33:10.332291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.336523+0000","last_change":"2023-12-11T19:32:11.586739+0000","last_active":"2023-12-11T19:33:10.336523+0000","last_peered":"2023-12-11T19:33:10.336523+0000","last_clean":"2023-12-11T19:33:10.336523+0000","last_became_active":"2023-12-11T19:32:11.586457+0000","last_became_peered":"2023-12-11T19:32:11.586457+0000","last_unstale":"2023-12-11T19:33:10.336523+0000","last_undegraded":"2023-12-11T19:33:10.336523+0000","last_fullsized":"2023-12-11T19:33:10.336523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:50.986234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341946+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:33:10.341946+0000","last_peered":"2023-12-11T19:33:10.341946+0000","last_clean":"2023-12-11T19:33:10.341946+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:33:10.341946+0000","last_undegraded":"2023-12-11T19:33:10.341946+0000","last_fullsized":"2023-12-11T19:33:10.341946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333225+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:33:10.333225+0000","last_peered":"2023-12-11T19:33:10.333225+0000","last_clean":"2023-12-11T19:33:10.333225+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:33:10.333225+0000","last_undegraded":"2023-12-11T19:33:10.333225+0000","last_fullsized":"2023-12-11T19:33:10.333225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339819+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:33:10.339819+0000","last_peered":"2023-12-11T19:33:10.339819+0000","last_clean":"2023-12-11T19:33:10.339819+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:33:10.339819+0000","last_undegraded":"2023-12-11T19:33:10.339819+0000","last_fullsized":"2023-12-11T19:33:10.339819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.319673+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:33:10.319673+0000","last_peered":"2023-12-11T19:33:10.319673+0000","last_clean":"2023-12-11T19:33:10.319673+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:33:10.319673+0000","last_undegraded":"2023-12-11T19:33:10.319673+0000","last_fullsized":"2023-12-11T19:33:10.319673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331108+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:33:10.331108+0000","last_peered":"2023-12-11T19:33:10.331108+0000","last_clean":"2023-12-11T19:33:10.331108+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:33:10.331108+0000","last_undegraded":"2023-12-11T19:33:10.331108+0000","last_fullsized":"2023-12-11T19:33:10.331108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337463+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:33:10.337463+0000","last_peered":"2023-12-11T19:33:10.337463+0000","last_clean":"2023-12-11T19:33:10.337463+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:33:10.337463+0000","last_undegraded":"2023-12-11T19:33:10.337463+0000","last_fullsized":"2023-12-11T19:33:10.337463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343501+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:33:10.343501+0000","last_peered":"2023-12-11T19:33:10.343501+0000","last_clean":"2023-12-11T19:33:10.343501+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:33:10.343501+0000","last_undegraded":"2023-12-11T19:33:10.343501+0000","last_fullsized":"2023-12-11T19:33:10.343501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341666+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:33:10.341666+0000","last_peered":"2023-12-11T19:33:10.341666+0000","last_clean":"2023-12-11T19:33:10.341666+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:33:10.341666+0000","last_undegraded":"2023-12-11T19:33:10.341666+0000","last_fullsized":"2023-12-11T19:33:10.341666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337779+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:33:10.337779+0000","last_peered":"2023-12-11T19:33:10.337779+0000","last_clean":"2023-12-11T19:33:10.337779+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:33:10.337779+0000","last_undegraded":"2023-12-11T19:33:10.337779+0000","last_fullsized":"2023-12-11T19:33:10.337779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70230,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1045,"num_read_kb":608,"num_write":255,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2181,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":210690,"data_compressed":5496,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":460267,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.94399999999999995,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80300000000000005,"5min":1.454,"15min":1.454},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95299999999999996,"5min":1.331,"15min":1.331},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.65800000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.077,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96599999999999997,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93400000000000005,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.60399999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.052,"5min":1.518,"15min":1.518},"last":0.80700000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.85099999999999998,"15min":0.871},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0720000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.85499999999999998,"15min":0.873},"min":{"1min":0.58399999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0980000000000001,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.77900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.115,"5min":1.403,"15min":1.403},"last":1},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.64200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88400000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.86199999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.69899999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1419999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.79200000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.165,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.67500000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0629999999999999,"5min":1.49,"15min":1.49},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.76100000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1819999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.90000000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1112588,"data_compressed":43838,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.8120000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.36899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91000000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.57599999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.96099999999999997,"5min":1.04,"15min":1.04},"last":0.87},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.032,"15min":1.032},"last":0.93700000000000006}]},{"osd":3,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.24},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95299999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.66600000000000004,"15min":0.70199999999999996},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76700000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.81100000000000005},"min":{"1min":0.61699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98399999999999999,"5min":1.085,"15min":1.085},"last":0.90600000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.58999999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97099999999999997,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89800000000000002,"15min":0.90400000000000003},"min":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.89400000000000002,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.626,"15min":0.626},"max":{"1min":0.995,"5min":1.643,"15min":1.643},"last":1.026}]},{"osd":9,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.69499999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0629999999999999,"5min":1.321,"15min":1.321},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.871,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.98899999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.97499999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.873},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1259999999999999,"5min":1.224,"15min":1.224},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90700000000000003,"15min":0.90200000000000002},"min":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1379999999999999,"5min":1.161,"15min":1.161},"last":0.99099999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":459822,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68899999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.101,"5min":6.101,"15min":6.101},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.76700000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.85199999999999998,"15min":0.84399999999999997},"min":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0589999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":6.0510000000000002,"5min":6.0510000000000002,"15min":6.0510000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.82899999999999996,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.024,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.875,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.278},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88,"15min":0.90800000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0429999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90500000000000003,"15min":0.93200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.389,"15min":1.389},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.92200000000000004,"15min":0.96799999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0129999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.94699999999999995,"15min":0.97699999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.081,"5min":1.274,"15min":1.274},"last":0.79900000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.95599999999999996,"15min":1.0029999999999999},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97499999999999998,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.97699999999999998,"15min":1.014},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.0309999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.93899999999999995,"15min":0.99099999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.0089999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.85399999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.61799999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":460077,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":1.097,"15min":1.097},"last":0.60999999999999999}]},{"osd":1,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.55700000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1479999999999999,"5min":1.351,"15min":1.351},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.81200000000000006,"15min":0.80900000000000005},"min":{"1min":0.61399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.0740000000000001,"5min":2.21,"15min":2.21},"last":0.86699999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.292,"15min":1.292},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.2350000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.876}]},{"osd":5,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.58199999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1140000000000001,"5min":2.173,"15min":2.173},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.083,"5min":1.177,"15min":1.177},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.84899999999999998,"15min":0.82099999999999995},"min":{"1min":0.74199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2869999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.022},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1459999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.71699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.304,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.74099999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.86599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.256,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.01,"15min":0.97499999999999998},"min":{"1min":0.94099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3600000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.113}]},{"osd":10,"last update":"Mon Dec 11 19:33:39 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.94299999999999995,"15min":0.89300000000000002},"min":{"1min":0.68400000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.246,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.103},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0169999999999999,"15min":0.98199999999999998},"min":{"1min":0.88100000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.3460000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.98299999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:32:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.1319999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1054433,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.51600000000000001,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76900000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59599999999999997},"min":{"1min":0.35399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.165},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77700000000000002,"15min":0.77000000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.89000000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.754,"15min":0.80600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97199999999999998,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70399999999999996,"15min":0.73799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59299999999999997,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73199999999999998,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.042,"5min":1.238,"15min":1.238},"last":0.626}]},{"osd":5,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96899999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.026,"5min":1.28,"15min":1.28},"last":0.998}]},{"osd":6,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79900000000000004,"15min":0.85499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.512,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.751,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.83699999999999997,"5min":1.359,"15min":1.359},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97999999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.85999999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.872}]},{"osd":9,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.98399999999999999,"5min":1.296,"15min":1.296},"last":0.753}]},{"osd":10,"last update":"Mon Dec 11 19:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.90700000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0369999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.194,"5min":1.25,"15min":1.25},"last":0.77700000000000002}]},{"osd":11,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.028},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.94499999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459920,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48299999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85999999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.046,"5min":1.089,"15min":1.089},"last":0.80700000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73099999999999998,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0269999999999999,"5min":1.284,"15min":1.284},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.494,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80000000000000004},"min":{"1min":0.58099999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.0860000000000001,"5min":1.157,"15min":1.157},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.84099999999999997,"15min":0.81799999999999995},"min":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86899999999999999,"15min":0.91600000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.603,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88500000000000001,"15min":0.87},"min":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.137,"5min":1.234,"15min":1.234},"last":0.93100000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":0.99399999999999999,"5min":1.327,"15min":1.327},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.871,"15min":0.88100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.375,"5min":1.379,"15min":1.379},"last":0.79100000000000004}]},{"osd":9,"last update":"Mon Dec 11 19:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.90900000000000003,"15min":0.91700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3220000000000001,"5min":1.345,"15min":1.345},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.876}]},{"osd":10,"last update":"Mon Dec 11 19:33:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83599999999999997,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.94699999999999995,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.96699999999999997,"15min":0.97099999999999997},"min":{"1min":0.70999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.081,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522978,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.55700000000000005},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69099999999999995,"15min":0.58199999999999996},"min":{"1min":0.58099999999999996,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":1.05,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.79900000000000004,"15min":0.72099999999999997},"min":{"1min":0.67600000000000005,"5min":0.42399999999999999,"15min":0.38400000000000001},"max":{"1min":1.1399999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.752,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.435,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.99299999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.81100000000000005,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":14.044,"5min":14.044,"15min":14.044},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80600000000000005,"5min":1.022,"15min":1.022},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.66600000000000004,"15min":0.57199999999999995},"min":{"1min":0.61299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.083,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Mon Dec 11 19:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.749,"15min":0.68100000000000005},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1539999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.73599999999999999,"15min":0.85599999999999998},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93600000000000005,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":1.01}]},{"osd":6,"last update":"Mon Dec 11 19:32:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.77800000000000002,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.627,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.997}]},{"osd":7,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.74199999999999999,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.68000000000000005,"15min":0.627},"min":{"1min":0.501,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79300000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2110000000000001,"5min":1.52,"15min":1.52},"last":1.127},{"interface":"front","average":{"1min":1.091,"5min":0.88500000000000001,"15min":0.83799999999999997},"min":{"1min":0.67100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.1419999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:10 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.91700000000000004,"15min":0.875},"min":{"1min":0.77400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.343,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.008,"5min":0.91800000000000004,"15min":0.88400000000000001},"min":{"1min":0.81200000000000006,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.105}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.127,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.008,"5min":0.96499999999999997,"15min":0.93899999999999995},"min":{"1min":0.79700000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.155}]},{"osd":11,"last update":"Mon Dec 11 19:32:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0509999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":461253,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82799999999999996,"15min":0.67800000000000005},"min":{"1min":0.52300000000000002,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":1.1890000000000001,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":1.0449999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.748,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0189999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.41399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.403},"last":0.63500000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71899999999999997,"15min":0.67800000000000005},"min":{"1min":0.55100000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.64900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3799999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79400000000000004,"15min":0.81799999999999995},"min":{"1min":0.58599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.4099999999999999,"5min":1.819,"15min":1.819},"last":1.0209999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.081,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.038},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.80700000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.90400000000000003,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1579999999999999,"5min":1.25,"15min":1.25},"last":0.98799999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96399999999999997,"15min":1.0069999999999999},"min":{"1min":0.57099999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92000000000000004,"15min":0.95899999999999996},"min":{"1min":0.58699999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.85199999999999998}]},{"osd":10,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.085},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.88900000000000001}]},{"osd":11,"last update":"Mon Dec 11 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.90100000000000002,"15min":0.88100000000000001},"min":{"1min":0.82099999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1850000000000001,"5min":0.95799999999999996,"15min":0.92000000000000004},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":6.9299999999999997,"5min":6.9299999999999997,"15min":6.9299999999999997},"last":1.103}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":519513,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.753,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.95199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71599999999999997,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":1.0760000000000001,"5min":1.115,"15min":1.115},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.64800000000000002},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Dec 11 19:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63700000000000001,"15min":0.60399999999999998},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.65200000000000002},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Mon Dec 11 19:32:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.93799999999999994}]},{"osd":5,"last update":"Mon Dec 11 19:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.97399999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86499999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Dec 11 19:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.747,"15min":0.70099999999999996},"min":{"1min":0.61799999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76800000000000002,"15min":0.747},"min":{"1min":0.64000000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0660000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.751},"min":{"1min":0.59499999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.119,"5min":1.133,"15min":1.133},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.86099999999999999,"15min":0.83299999999999996},"min":{"1min":0.70299999999999996,"5min":0.627,"15min":0.627},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.86599999999999999},"min":{"1min":0.68000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.137,"5min":1.167,"15min":1.167},"last":0.92100000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.91400000000000003,"15min":0.89500000000000002},"min":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.095,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91500000000000004,"15min":0.88900000000000001},"min":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.109,"5min":1.169,"15min":1.169},"last":0.97799999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85399999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":458799,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48499999999999999,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":0.80700000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.28599999999999998},"max":{"1min":0.93400000000000005,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.625},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98999999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.93999999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.64700000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.96999999999999997,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.65100000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.36199999999999999},"max":{"1min":1.0660000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88}]},{"osd":4,"last update":"Mon Dec 11 19:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.751,"15min":0.66700000000000004},"min":{"1min":0.57599999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.80500000000000005,"15min":0.71699999999999997},"min":{"1min":0.65300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.91000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.198,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77000000000000002,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1220000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.006}]},{"osd":6,"last update":"Mon Dec 11 19:32:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.196,"5min":1.28,"15min":1.28},"last":0.746}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86499999999999999,"15min":0.82699999999999996},"min":{"1min":0.68600000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1399999999999999,"5min":1.752,"15min":1.752},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86499999999999999,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.206,"5min":1.631,"15min":1.631},"last":1.0620000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93899999999999995,"15min":0.96399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.042},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.95899999999999996,"15min":1.0029999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3220000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.95399999999999996,"15min":0.97299999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.98499999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.111},{"interface":"front","average":{"1min":0.88700000000000001,"5min":1.0189999999999999,"15min":1.0549999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1000000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.095}]},{"osd":11,"last update":"Mon Dec 11 19:32:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.141},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0269999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461955,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.496},"min":{"1min":0.39400000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85199999999999998,"5min":1.294,"15min":1.294},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90500000000000003,"5min":1.018,"15min":1.018},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.58899999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0209999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.98699999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69999999999999996,"15min":0.65900000000000003},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0680000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.877},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.76900000000000002,"15min":0.70799999999999996},"min":{"1min":0.57799999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0620000000000001,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.71999999999999997},"min":{"1min":0.51800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Mon Dec 11 19:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72999999999999998,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.395,"5min":1.429,"15min":1.429},"last":0.94199999999999995}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82599999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.997,"5min":1.26,"15min":1.26},"last":0.96199999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.67600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0369999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.109,"5min":1.494,"15min":1.494},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.81799999999999995,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1100000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94399999999999995,"15min":0.95499999999999996},"min":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0960000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Mon Dec 11 19:32:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.999}]}]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1052916,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.624},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83799999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.65200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83899999999999997,"5min":1.177,"15min":1.177},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.53000000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71999999999999997,"15min":0.69199999999999995},"min":{"1min":0.56299999999999994,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98999999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.47099999999999997},"min":{"1min":0.372,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.879,"15min":0.879},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94499999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52400000000000002}]},{"osd":3,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.74199999999999999},"min":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96699999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.83399999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96499999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91000000000000003,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.58799999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.745},"min":{"1min":0.55500000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1240000000000001,"5min":1.597,"15min":1.597},"last":0.66500000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78000000000000003,"15min":0.75},"min":{"1min":0.58799999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.751},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.83999999999999997,"15min":0.80900000000000005},"min":{"1min":0.71599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0760000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.214,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.873},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77200000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83899999999999997},"min":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.88100000000000001,"15min":0.88800000000000001},"min":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2509999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Mon Dec 11 19:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91500000000000004,"15min":0.90900000000000003},"min":{"1min":0.81399999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1060000000000001,"5min":1.548,"15min":1.548},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.002,"5min":0.94499999999999995,"15min":0.93100000000000005},"min":{"1min":0.77200000000000002,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64674,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60936,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63520,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:33:42.541 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-12-11T19:33:42.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/840715367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 msgr2=0x7f3c281098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/840715367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281098e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000b0d0 tx=0x7f3c2000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/840715367 shutdown_connections 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/840715367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c281014a0 0x7f3c281018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/840715367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281098e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/840715367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c28102690 0x7f3c28102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/840715367 >> 172.21.15.66:0/840715367 conn(0x7f3c280fca10 msgr2=0x7f3c280fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.782+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/840715367 shutdown_connections 2023-12-11T19:33:42.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.783+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/840715367 wait complete. 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 Processor -- start 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 -- start start 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c281014a0 0x7f3c281996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c28102690 0x7f3c28199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c281a1160 con 0x7f3c28103030 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c281a12a0 con 0x7f3c281014a0 2023-12-11T19:33:42.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2d1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c281a1430 con 0x7f3c28102690 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2759e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2759e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50796/0 (socket says 172.21.15.66:50796) 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2759e700 1 -- 172.21.15.66:0/954968934 learned_addr learned my addr 172.21.15.66:0/954968934 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c26d9d700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c281014a0 0x7f3c281996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.784+0000 7f3c2659c700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c28102690 0x7f3c28199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2759e700 1 -- 172.21.15.66:0/954968934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c28102690 msgr2=0x7f3c28199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2759e700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c28102690 0x7f3c28199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2759e700 1 -- 172.21.15.66:0/954968934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c281014a0 msgr2=0x7f3c281996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2759e700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c281014a0 0x7f3c281996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2759e700 1 -- 172.21.15.66:0/954968934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c20009d00 con 0x7f3c28103030 2023-12-11T19:33:42.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2759e700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281a0b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3c200079c0 tx=0x7f3c2000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000e040 con 0x7f3c28103030 2023-12-11T19:33:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c2000ceb0 con 0x7f3c28103030 2023-12-11T19:33:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001b6b0 con 0x7f3c28103030 2023-12-11T19:33:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.785+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c281a16b0 con 0x7f3c28103030 2023-12-11T19:33:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.786+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c281a1ae0 con 0x7f3c28103030 2023-12-11T19:33:42.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.787+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c28068230 con 0x7f3c28103030 2023-12-11T19:33:42.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.789+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3c2001d480 con 0x7f3c28103030 2023-12-11T19:33:42.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.790+0000 7f3c0ffff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c100a1500 0x7f3c100a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:42.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.790+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c200cee50 con 0x7f3c28103030 2023-12-11T19:33:42.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.790+0000 7f3c26d9d700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c100a1500 0x7f3c100a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:42.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.791+0000 7f3c26d9d700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c100a1500 0x7f3c100a39c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c281024f0 tx=0x7f3c18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:42.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:42.792+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3c20098540 con 0x7f3c28103030 2023-12-11T19:33:43.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.008+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 --> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c28074f20 con 0x7f3c100a1500 2023-12-11T19:33:43.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.019+0000 7f3c0ffff700 1 -- 172.21.15.66:0/954968934 <== mgr.14150 v2:172.21.15.66:6800/775613503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343487 (secure 0 0 0) 0x7f3c28074f20 con 0x7f3c100a1500 2023-12-11T19:33:43.024 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c100a1500 msgr2=0x7f3c100a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c100a1500 0x7f3c100a39c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c281024f0 tx=0x7f3c18009260 comp rx=0 tx=0).stop 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 msgr2=0x7f3c281a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281a0b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3c200079c0 tx=0x7f3c2000cab0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 shutdown_connections 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c100a1500 0x7f3c100a39c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c281014a0 0x7f3c281996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c28103030 0x7f3c281a0b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 --2- 172.21.15.66:0/954968934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c28102690 0x7f3c28199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 >> 172.21.15.66:0/954968934 conn(0x7f3c280fca10 msgr2=0x7f3c28073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:43.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.024+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 shutdown_connections 2023-12-11T19:33:43.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.025+0000 7f3c2d1ff700 1 -- 172.21.15.66:0/954968934 wait complete. 2023-12-11T19:33:43.028 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-12-11T19:33:43.043 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2023-12-11T19:33:42.312135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660844,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1171,"num_read_kb":717,"num_write":438,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1183040,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335976,"kb_used_data":11784,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605848,"statfs":{"total":1151940427776,"available":1151596388352,"internally_reserved":0,"allocated":12066816,"data_stored":7484521,"data_compressed":183741,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2490,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":16,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002955"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343455+0000","last_change":"2023-12-11T19:31:52.275839+0000","last_active":"2023-12-11T19:33:10.343455+0000","last_peered":"2023-12-11T19:33:10.343455+0000","last_clean":"2023-12-11T19:33:10.343455+0000","last_became_active":"2023-12-11T19:31:52.275164+0000","last_became_peered":"2023-12-11T19:31:52.275164+0000","last_unstale":"2023-12-11T19:33:10.343455+0000","last_undegraded":"2023-12-11T19:33:10.343455+0000","last_fullsized":"2023-12-11T19:33:10.343455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:17:42.418590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.334250+0000","last_change":"2023-12-11T19:31:52.265916+0000","last_active":"2023-12-11T19:33:10.334250+0000","last_peered":"2023-12-11T19:33:10.334250+0000","last_clean":"2023-12-11T19:33:10.334250+0000","last_became_active":"2023-12-11T19:31:52.264891+0000","last_became_peered":"2023-12-11T19:31:52.264891+0000","last_unstale":"2023-12-11T19:33:10.334250+0000","last_undegraded":"2023-12-11T19:33:10.334250+0000","last_fullsized":"2023-12-11T19:33:10.334250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:07.470545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340675+0000","last_change":"2023-12-11T19:31:52.273700+0000","last_active":"2023-12-11T19:33:10.340675+0000","last_peered":"2023-12-11T19:33:10.340675+0000","last_clean":"2023-12-11T19:33:10.340675+0000","last_became_active":"2023-12-11T19:31:52.272374+0000","last_became_peered":"2023-12-11T19:31:52.272374+0000","last_unstale":"2023-12-11T19:33:10.340675+0000","last_undegraded":"2023-12-11T19:33:10.340675+0000","last_fullsized":"2023-12-11T19:33:10.340675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:45:40.857978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.336763+0000","last_change":"2023-12-11T19:31:52.279408+0000","last_active":"2023-12-11T19:33:10.336763+0000","last_peered":"2023-12-11T19:33:10.336763+0000","last_clean":"2023-12-11T19:33:10.336763+0000","last_became_active":"2023-12-11T19:31:52.278842+0000","last_became_peered":"2023-12-11T19:31:52.278842+0000","last_unstale":"2023-12-11T19:33:10.336763+0000","last_undegraded":"2023-12-11T19:33:10.336763+0000","last_fullsized":"2023-12-11T19:33:10.336763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:10.310775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324900+0000","last_change":"2023-12-11T19:31:52.268580+0000","last_active":"2023-12-11T19:33:10.324900+0000","last_peered":"2023-12-11T19:33:10.324900+0000","last_clean":"2023-12-11T19:33:10.324900+0000","last_became_active":"2023-12-11T19:31:52.268282+0000","last_became_peered":"2023-12-11T19:31:52.268282+0000","last_unstale":"2023-12-11T19:33:10.324900+0000","last_undegraded":"2023-12-11T19:33:10.324900+0000","last_fullsized":"2023-12-11T19:33:10.324900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:48:48.953083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324677+0000","last_change":"2023-12-11T19:32:11.585539+0000","last_active":"2023-12-11T19:33:10.324677+0000","last_peered":"2023-12-11T19:33:10.324677+0000","last_clean":"2023-12-11T19:33:10.324677+0000","last_became_active":"2023-12-11T19:32:11.585087+0000","last_became_peered":"2023-12-11T19:32:11.585087+0000","last_unstale":"2023-12-11T19:33:10.324677+0000","last_undegraded":"2023-12-11T19:33:10.324677+0000","last_fullsized":"2023-12-11T19:33:10.324677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:23.076166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332857+0000","last_change":"2023-12-11T19:31:52.274775+0000","last_active":"2023-12-11T19:33:10.332857+0000","last_peered":"2023-12-11T19:33:10.332857+0000","last_clean":"2023-12-11T19:33:10.332857+0000","last_became_active":"2023-12-11T19:31:52.273274+0000","last_became_peered":"2023-12-11T19:31:52.273274+0000","last_unstale":"2023-12-11T19:33:10.332857+0000","last_undegraded":"2023-12-11T19:33:10.332857+0000","last_fullsized":"2023-12-11T19:33:10.332857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:06.849848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341918+0000","last_change":"2023-12-11T19:31:52.280382+0000","last_active":"2023-12-11T19:33:10.341918+0000","last_peered":"2023-12-11T19:33:10.341918+0000","last_clean":"2023-12-11T19:33:10.341918+0000","last_became_active":"2023-12-11T19:31:52.279802+0000","last_became_peered":"2023-12-11T19:31:52.279802+0000","last_unstale":"2023-12-11T19:33:10.341918+0000","last_undegraded":"2023-12-11T19:33:10.341918+0000","last_fullsized":"2023-12-11T19:33:10.341918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:34:18.689826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343323+0000","last_change":"2023-12-11T19:31:52.264616+0000","last_active":"2023-12-11T19:33:10.343323+0000","last_peered":"2023-12-11T19:33:10.343323+0000","last_clean":"2023-12-11T19:33:10.343323+0000","last_became_active":"2023-12-11T19:31:52.264256+0000","last_became_peered":"2023-12-11T19:31:52.264256+0000","last_unstale":"2023-12-11T19:33:10.343323+0000","last_undegraded":"2023-12-11T19:33:10.343323+0000","last_fullsized":"2023-12-11T19:33:10.343323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:51:49.740579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339811+0000","last_change":"2023-12-11T19:31:52.269240+0000","last_active":"2023-12-11T19:33:10.339811+0000","last_peered":"2023-12-11T19:33:10.339811+0000","last_clean":"2023-12-11T19:33:10.339811+0000","last_became_active":"2023-12-11T19:31:52.268563+0000","last_became_peered":"2023-12-11T19:31:52.268563+0000","last_unstale":"2023-12-11T19:33:10.339811+0000","last_undegraded":"2023-12-11T19:33:10.339811+0000","last_fullsized":"2023-12-11T19:33:10.339811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:37.329075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339748+0000","last_change":"2023-12-11T19:31:52.276407+0000","last_active":"2023-12-11T19:33:10.339748+0000","last_peered":"2023-12-11T19:33:10.339748+0000","last_clean":"2023-12-11T19:33:10.339748+0000","last_became_active":"2023-12-11T19:31:52.275992+0000","last_became_peered":"2023-12-11T19:31:52.275992+0000","last_unstale":"2023-12-11T19:33:10.339748+0000","last_undegraded":"2023-12-11T19:33:10.339748+0000","last_fullsized":"2023-12-11T19:33:10.339748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:28:04.691576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331280+0000","last_change":"2023-12-11T19:31:52.275968+0000","last_active":"2023-12-11T19:33:10.331280+0000","last_peered":"2023-12-11T19:33:10.331280+0000","last_clean":"2023-12-11T19:33:10.331280+0000","last_became_active":"2023-12-11T19:31:52.275428+0000","last_became_peered":"2023-12-11T19:31:52.275428+0000","last_unstale":"2023-12-11T19:33:10.331280+0000","last_undegraded":"2023-12-11T19:33:10.331280+0000","last_fullsized":"2023-12-11T19:33:10.331280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:53:32.380207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337449+0000","last_change":"2023-12-11T19:31:52.269138+0000","last_active":"2023-12-11T19:33:10.337449+0000","last_peered":"2023-12-11T19:33:10.337449+0000","last_clean":"2023-12-11T19:33:10.337449+0000","last_became_active":"2023-12-11T19:31:52.268590+0000","last_became_peered":"2023-12-11T19:31:52.268590+0000","last_unstale":"2023-12-11T19:33:10.337449+0000","last_undegraded":"2023-12-11T19:33:10.337449+0000","last_fullsized":"2023-12-11T19:33:10.337449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:33:52.496489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.334371+0000","last_change":"2023-12-11T19:31:52.266696+0000","last_active":"2023-12-11T19:33:10.334371+0000","last_peered":"2023-12-11T19:33:10.334371+0000","last_clean":"2023-12-11T19:33:10.334371+0000","last_became_active":"2023-12-11T19:31:52.264613+0000","last_became_peered":"2023-12-11T19:31:52.264613+0000","last_unstale":"2023-12-11T19:33:10.334371+0000","last_undegraded":"2023-12-11T19:33:10.334371+0000","last_fullsized":"2023-12-11T19:33:10.334371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:04:11.893543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341068+0000","last_change":"2023-12-11T19:31:52.267001+0000","last_active":"2023-12-11T19:33:10.341068+0000","last_peered":"2023-12-11T19:33:10.341068+0000","last_clean":"2023-12-11T19:33:10.341068+0000","last_became_active":"2023-12-11T19:31:52.264158+0000","last_became_peered":"2023-12-11T19:31:52.264158+0000","last_unstale":"2023-12-11T19:33:10.341068+0000","last_undegraded":"2023-12-11T19:33:10.341068+0000","last_fullsized":"2023-12-11T19:33:10.341068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:12.921775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344589+0000","last_change":"2023-12-11T19:31:52.270518+0000","last_active":"2023-12-11T19:33:10.344589+0000","last_peered":"2023-12-11T19:33:10.344589+0000","last_clean":"2023-12-11T19:33:10.344589+0000","last_became_active":"2023-12-11T19:31:52.270198+0000","last_became_peered":"2023-12-11T19:31:52.270198+0000","last_unstale":"2023-12-11T19:33:10.344589+0000","last_undegraded":"2023-12-11T19:33:10.344589+0000","last_fullsized":"2023-12-11T19:33:10.344589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:45:23.772216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343527+0000","last_change":"2023-12-11T19:31:52.277337+0000","last_active":"2023-12-11T19:33:10.343527+0000","last_peered":"2023-12-11T19:33:10.343527+0000","last_clean":"2023-12-11T19:33:10.343527+0000","last_became_active":"2023-12-11T19:31:52.276611+0000","last_became_peered":"2023-12-11T19:31:52.276611+0000","last_unstale":"2023-12-11T19:33:10.343527+0000","last_undegraded":"2023-12-11T19:33:10.343527+0000","last_fullsized":"2023-12-11T19:33:10.343527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:17:23.897474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333632+0000","last_change":"2023-12-11T19:31:52.274486+0000","last_active":"2023-12-11T19:33:10.333632+0000","last_peered":"2023-12-11T19:33:10.333632+0000","last_clean":"2023-12-11T19:33:10.333632+0000","last_became_active":"2023-12-11T19:31:52.272398+0000","last_became_peered":"2023-12-11T19:31:52.272398+0000","last_unstale":"2023-12-11T19:33:10.333632+0000","last_undegraded":"2023-12-11T19:33:10.333632+0000","last_fullsized":"2023-12-11T19:33:10.333632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:44:16.449859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324560+0000","last_change":"2023-12-11T19:31:52.272229+0000","last_active":"2023-12-11T19:33:10.324560+0000","last_peered":"2023-12-11T19:33:10.324560+0000","last_clean":"2023-12-11T19:33:10.324560+0000","last_became_active":"2023-12-11T19:31:52.271205+0000","last_became_peered":"2023-12-11T19:31:52.271205+0000","last_unstale":"2023-12-11T19:33:10.324560+0000","last_undegraded":"2023-12-11T19:33:10.324560+0000","last_fullsized":"2023-12-11T19:33:10.324560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:20:07.236360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":130,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:37.707150+0000","last_change":"2023-12-11T19:31:51.259726+0000","last_active":"2023-12-11T19:33:37.707150+0000","last_peered":"2023-12-11T19:33:37.707150+0000","last_clean":"2023-12-11T19:33:37.707150+0000","last_became_active":"2023-12-11T19:31:51.259259+0000","last_became_peered":"2023-12-11T19:31:51.259259+0000","last_unstale":"2023-12-11T19:33:37.707150+0000","last_undegraded":"2023-12-11T19:33:37.707150+0000","last_fullsized":"2023-12-11T19:33:37.707150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:04.800569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.328995+0000","last_change":"2023-12-11T19:31:52.277020+0000","last_active":"2023-12-11T19:33:10.328995+0000","last_peered":"2023-12-11T19:33:10.328995+0000","last_clean":"2023-12-11T19:33:10.328995+0000","last_became_active":"2023-12-11T19:31:52.276723+0000","last_became_peered":"2023-12-11T19:31:52.276723+0000","last_unstale":"2023-12-11T19:33:10.328995+0000","last_undegraded":"2023-12-11T19:33:10.328995+0000","last_fullsized":"2023-12-11T19:33:10.328995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:56:18.607590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586855+0000","last_change":"2023-12-11T19:31:51.264802+0000","last_active":"2023-12-11T19:32:10.586855+0000","last_peered":"2023-12-11T19:32:10.586855+0000","last_clean":"2023-12-11T19:32:10.586855+0000","last_became_active":"2023-12-11T19:31:51.264387+0000","last_became_peered":"2023-12-11T19:31:51.264387+0000","last_unstale":"2023-12-11T19:32:10.586855+0000","last_undegraded":"2023-12-11T19:32:10.586855+0000","last_fullsized":"2023-12-11T19:32:10.586855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:51:30.435361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344983+0000","last_change":"2023-12-11T19:31:52.278062+0000","last_active":"2023-12-11T19:33:10.344983+0000","last_peered":"2023-12-11T19:33:10.344983+0000","last_clean":"2023-12-11T19:33:10.344983+0000","last_became_active":"2023-12-11T19:31:52.276887+0000","last_became_peered":"2023-12-11T19:31:52.276887+0000","last_unstale":"2023-12-11T19:33:10.344983+0000","last_undegraded":"2023-12-11T19:33:10.344983+0000","last_fullsized":"2023-12-11T19:33:10.344983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:07:07.286850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585009+0000","last_change":"2023-12-11T19:31:51.262929+0000","last_active":"2023-12-11T19:32:10.585009+0000","last_peered":"2023-12-11T19:32:10.585009+0000","last_clean":"2023-12-11T19:32:10.585009+0000","last_became_active":"2023-12-11T19:31:51.262326+0000","last_became_peered":"2023-12-11T19:31:51.262326+0000","last_unstale":"2023-12-11T19:32:10.585009+0000","last_undegraded":"2023-12-11T19:32:10.585009+0000","last_fullsized":"2023-12-11T19:32:10.585009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:51.958882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343252+0000","last_change":"2023-12-11T19:31:52.267525+0000","last_active":"2023-12-11T19:33:10.343252+0000","last_peered":"2023-12-11T19:33:10.343252+0000","last_clean":"2023-12-11T19:33:10.343252+0000","last_became_active":"2023-12-11T19:31:52.266991+0000","last_became_peered":"2023-12-11T19:31:52.266991+0000","last_unstale":"2023-12-11T19:33:10.343252+0000","last_undegraded":"2023-12-11T19:33:10.343252+0000","last_fullsized":"2023-12-11T19:33:10.343252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:07:00.464063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584594+0000","last_change":"2023-12-11T19:31:51.261873+0000","last_active":"2023-12-11T19:32:10.584594+0000","last_peered":"2023-12-11T19:32:10.584594+0000","last_clean":"2023-12-11T19:32:10.584594+0000","last_became_active":"2023-12-11T19:31:51.260236+0000","last_became_peered":"2023-12-11T19:31:51.260236+0000","last_unstale":"2023-12-11T19:32:10.584594+0000","last_undegraded":"2023-12-11T19:32:10.584594+0000","last_fullsized":"2023-12-11T19:32:10.584594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:02.112928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339985+0000","last_change":"2023-12-11T19:31:52.271766+0000","last_active":"2023-12-11T19:33:10.339985+0000","last_peered":"2023-12-11T19:33:10.339985+0000","last_clean":"2023-12-11T19:33:10.339985+0000","last_became_active":"2023-12-11T19:31:52.271431+0000","last_became_peered":"2023-12-11T19:31:52.271431+0000","last_unstale":"2023-12-11T19:33:10.339985+0000","last_undegraded":"2023-12-11T19:33:10.339985+0000","last_fullsized":"2023-12-11T19:33:10.339985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:23:53.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:37.733788+0000","last_change":"2023-12-11T19:32:11.601552+0000","last_active":"2023-12-11T19:33:37.733788+0000","last_peered":"2023-12-11T19:33:37.733788+0000","last_clean":"2023-12-11T19:33:37.733788+0000","last_became_active":"2023-12-11T19:32:11.593828+0000","last_became_peered":"2023-12-11T19:32:11.593828+0000","last_unstale":"2023-12-11T19:33:37.733788+0000","last_undegraded":"2023-12-11T19:33:37.733788+0000","last_fullsized":"2023-12-11T19:33:37.733788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:53.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1650,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333342+0000","last_change":"2023-12-11T19:31:52.274438+0000","last_active":"2023-12-11T19:33:10.333342+0000","last_peered":"2023-12-11T19:33:10.333342+0000","last_clean":"2023-12-11T19:33:10.333342+0000","last_became_active":"2023-12-11T19:31:52.274118+0000","last_became_peered":"2023-12-11T19:31:52.274118+0000","last_unstale":"2023-12-11T19:33:10.333342+0000","last_undegraded":"2023-12-11T19:33:10.333342+0000","last_fullsized":"2023-12-11T19:33:10.333342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:36:46.964042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.586092+0000","last_change":"2023-12-11T19:31:51.255976+0000","last_active":"2023-12-11T19:32:10.586092+0000","last_peered":"2023-12-11T19:32:10.586092+0000","last_clean":"2023-12-11T19:32:10.586092+0000","last_became_active":"2023-12-11T19:31:51.255601+0000","last_became_peered":"2023-12-11T19:31:51.255601+0000","last_unstale":"2023-12-11T19:32:10.586092+0000","last_undegraded":"2023-12-11T19:32:10.586092+0000","last_fullsized":"2023-12-11T19:32:10.586092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:07:54.656039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.338447+0000","last_change":"2023-12-11T19:31:52.270369+0000","last_active":"2023-12-11T19:33:10.338447+0000","last_peered":"2023-12-11T19:33:10.338447+0000","last_clean":"2023-12-11T19:33:10.338447+0000","last_became_active":"2023-12-11T19:31:52.269988+0000","last_became_peered":"2023-12-11T19:31:52.269988+0000","last_unstale":"2023-12-11T19:33:10.338447+0000","last_undegraded":"2023-12-11T19:33:10.338447+0000","last_fullsized":"2023-12-11T19:33:10.338447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:16:13.070256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581195+0000","last_change":"2023-12-11T19:31:51.245236+0000","last_active":"2023-12-11T19:32:10.581195+0000","last_peered":"2023-12-11T19:32:10.581195+0000","last_clean":"2023-12-11T19:32:10.581195+0000","last_became_active":"2023-12-11T19:31:51.243002+0000","last_became_peered":"2023-12-11T19:31:51.243002+0000","last_unstale":"2023-12-11T19:32:10.581195+0000","last_undegraded":"2023-12-11T19:32:10.581195+0000","last_fullsized":"2023-12-11T19:32:10.581195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:41:47.019500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337715+0000","last_change":"2023-12-11T19:31:52.275108+0000","last_active":"2023-12-11T19:33:10.337715+0000","last_peered":"2023-12-11T19:33:10.337715+0000","last_clean":"2023-12-11T19:33:10.337715+0000","last_became_active":"2023-12-11T19:31:52.274633+0000","last_became_peered":"2023-12-11T19:31:52.274633+0000","last_unstale":"2023-12-11T19:33:10.337715+0000","last_undegraded":"2023-12-11T19:33:10.337715+0000","last_fullsized":"2023-12-11T19:33:10.337715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:54:45.628779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581346+0000","last_change":"2023-12-11T19:31:51.253867+0000","last_active":"2023-12-11T19:32:10.581346+0000","last_peered":"2023-12-11T19:32:10.581346+0000","last_clean":"2023-12-11T19:32:10.581346+0000","last_became_active":"2023-12-11T19:31:51.253513+0000","last_became_peered":"2023-12-11T19:31:51.253513+0000","last_unstale":"2023-12-11T19:32:10.581346+0000","last_undegraded":"2023-12-11T19:32:10.581346+0000","last_fullsized":"2023-12-11T19:32:10.581346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:35:57.393454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342744+0000","last_change":"2023-12-11T19:31:52.269789+0000","last_active":"2023-12-11T19:33:10.342744+0000","last_peered":"2023-12-11T19:33:10.342744+0000","last_clean":"2023-12-11T19:33:10.342744+0000","last_became_active":"2023-12-11T19:31:52.269334+0000","last_became_peered":"2023-12-11T19:31:52.269334+0000","last_unstale":"2023-12-11T19:33:10.342744+0000","last_undegraded":"2023-12-11T19:33:10.342744+0000","last_fullsized":"2023-12-11T19:33:10.342744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:06:13.696876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":227,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.732269+0000","last_change":"2023-12-11T19:31:51.255958+0000","last_active":"2023-12-11T19:33:39.732269+0000","last_peered":"2023-12-11T19:33:39.732269+0000","last_clean":"2023-12-11T19:33:39.732269+0000","last_became_active":"2023-12-11T19:31:51.255515+0000","last_became_peered":"2023-12-11T19:31:51.255515+0000","last_unstale":"2023-12-11T19:33:39.732269+0000","last_undegraded":"2023-12-11T19:33:39.732269+0000","last_fullsized":"2023-12-11T19:33:39.732269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:41:12.750081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340936+0000","last_change":"2023-12-11T19:31:52.266733+0000","last_active":"2023-12-11T19:33:10.340936+0000","last_peered":"2023-12-11T19:33:10.340936+0000","last_clean":"2023-12-11T19:33:10.340936+0000","last_became_active":"2023-12-11T19:31:52.266399+0000","last_became_peered":"2023-12-11T19:31:52.266399+0000","last_unstale":"2023-12-11T19:33:10.340936+0000","last_undegraded":"2023-12-11T19:33:10.340936+0000","last_fullsized":"2023-12-11T19:33:10.340936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:50:40.231381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578707+0000","last_change":"2023-12-11T19:31:51.262453+0000","last_active":"2023-12-11T19:32:10.578707+0000","last_peered":"2023-12-11T19:32:10.578707+0000","last_clean":"2023-12-11T19:32:10.578707+0000","last_became_active":"2023-12-11T19:31:51.262071+0000","last_became_peered":"2023-12-11T19:31:51.262071+0000","last_unstale":"2023-12-11T19:32:10.578707+0000","last_undegraded":"2023-12-11T19:32:10.578707+0000","last_fullsized":"2023-12-11T19:32:10.578707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:04:17.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.340428+0000","last_change":"2023-12-11T19:31:52.272827+0000","last_active":"2023-12-11T19:33:10.340428+0000","last_peered":"2023-12-11T19:33:10.340428+0000","last_clean":"2023-12-11T19:33:10.340428+0000","last_became_active":"2023-12-11T19:31:52.271904+0000","last_became_peered":"2023-12-11T19:31:52.271904+0000","last_unstale":"2023-12-11T19:33:10.340428+0000","last_undegraded":"2023-12-11T19:33:10.340428+0000","last_fullsized":"2023-12-11T19:33:10.340428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:10:08.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581870+0000","last_change":"2023-12-11T19:31:51.245006+0000","last_active":"2023-12-11T19:32:10.581870+0000","last_peered":"2023-12-11T19:32:10.581870+0000","last_clean":"2023-12-11T19:32:10.581870+0000","last_became_active":"2023-12-11T19:31:51.244607+0000","last_became_peered":"2023-12-11T19:31:51.244607+0000","last_unstale":"2023-12-11T19:32:10.581870+0000","last_undegraded":"2023-12-11T19:32:10.581870+0000","last_fullsized":"2023-12-11T19:32:10.581870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:08:16.592150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344002+0000","last_change":"2023-12-11T19:31:52.271592+0000","last_active":"2023-12-11T19:33:10.344002+0000","last_peered":"2023-12-11T19:33:10.344002+0000","last_clean":"2023-12-11T19:33:10.344002+0000","last_became_active":"2023-12-11T19:31:52.267903+0000","last_became_peered":"2023-12-11T19:31:52.267903+0000","last_unstale":"2023-12-11T19:33:10.344002+0000","last_undegraded":"2023-12-11T19:33:10.344002+0000","last_fullsized":"2023-12-11T19:33:10.344002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:09:41.919353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":267,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.658525+0000","last_change":"2023-12-11T19:31:51.252199+0000","last_active":"2023-12-11T19:33:39.658525+0000","last_peered":"2023-12-11T19:33:39.658525+0000","last_clean":"2023-12-11T19:33:39.658525+0000","last_became_active":"2023-12-11T19:31:51.251806+0000","last_became_peered":"2023-12-11T19:31:51.251806+0000","last_unstale":"2023-12-11T19:33:39.658525+0000","last_undegraded":"2023-12-11T19:33:39.658525+0000","last_fullsized":"2023-12-11T19:33:39.658525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:47.515562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60756,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":168,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333830+0000","last_change":"2023-12-11T19:31:52.267666+0000","last_active":"2023-12-11T19:33:10.333830+0000","last_peered":"2023-12-11T19:33:10.333830+0000","last_clean":"2023-12-11T19:33:10.333830+0000","last_became_active":"2023-12-11T19:31:52.267350+0000","last_became_peered":"2023-12-11T19:31:52.267350+0000","last_unstale":"2023-12-11T19:33:10.333830+0000","last_undegraded":"2023-12-11T19:33:10.333830+0000","last_fullsized":"2023-12-11T19:33:10.333830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:01:40.141049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.587382+0000","last_change":"2023-12-11T19:31:51.259156+0000","last_active":"2023-12-11T19:32:10.587382+0000","last_peered":"2023-12-11T19:32:10.587382+0000","last_clean":"2023-12-11T19:32:10.587382+0000","last_became_active":"2023-12-11T19:31:51.256868+0000","last_became_peered":"2023-12-11T19:31:51.256868+0000","last_unstale":"2023-12-11T19:32:10.587382+0000","last_undegraded":"2023-12-11T19:32:10.587382+0000","last_fullsized":"2023-12-11T19:32:10.587382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:31:12.735587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337785+0000","last_change":"2023-12-11T19:31:52.274959+0000","last_active":"2023-12-11T19:33:10.337785+0000","last_peered":"2023-12-11T19:33:10.337785+0000","last_clean":"2023-12-11T19:33:10.337785+0000","last_became_active":"2023-12-11T19:31:52.273991+0000","last_became_peered":"2023-12-11T19:31:52.273991+0000","last_unstale":"2023-12-11T19:33:10.337785+0000","last_undegraded":"2023-12-11T19:33:10.337785+0000","last_fullsized":"2023-12-11T19:33:10.337785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:33:08.963883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581282+0000","last_change":"2023-12-11T19:31:51.261077+0000","last_active":"2023-12-11T19:32:10.581282+0000","last_peered":"2023-12-11T19:32:10.581282+0000","last_clean":"2023-12-11T19:32:10.581282+0000","last_became_active":"2023-12-11T19:31:51.260657+0000","last_became_peered":"2023-12-11T19:31:51.260657+0000","last_unstale":"2023-12-11T19:32:10.581282+0000","last_undegraded":"2023-12-11T19:32:10.581282+0000","last_fullsized":"2023-12-11T19:32:10.581282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:20:47.403233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344461+0000","last_change":"2023-12-11T19:31:52.278853+0000","last_active":"2023-12-11T19:33:10.344461+0000","last_peered":"2023-12-11T19:33:10.344461+0000","last_clean":"2023-12-11T19:33:10.344461+0000","last_became_active":"2023-12-11T19:31:52.278531+0000","last_became_peered":"2023-12-11T19:31:52.278531+0000","last_unstale":"2023-12-11T19:33:10.344461+0000","last_undegraded":"2023-12-11T19:33:10.344461+0000","last_fullsized":"2023-12-11T19:33:10.344461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:08:10.739138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'8","reported_seq":126,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:36.732277+0000","last_change":"2023-12-11T19:31:51.252893+0000","last_active":"2023-12-11T19:33:36.732277+0000","last_peered":"2023-12-11T19:33:36.732277+0000","last_clean":"2023-12-11T19:33:36.732277+0000","last_became_active":"2023-12-11T19:31:51.252514+0000","last_became_peered":"2023-12-11T19:31:51.252514+0000","last_unstale":"2023-12-11T19:33:36.732277+0000","last_undegraded":"2023-12-11T19:33:36.732277+0000","last_fullsized":"2023-12-11T19:33:36.732277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:54:22.817609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1616,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341155+0000","last_change":"2023-12-11T19:31:52.277362+0000","last_active":"2023-12-11T19:33:10.341155+0000","last_peered":"2023-12-11T19:33:10.341155+0000","last_clean":"2023-12-11T19:33:10.341155+0000","last_became_active":"2023-12-11T19:31:52.276260+0000","last_became_peered":"2023-12-11T19:31:52.276260+0000","last_unstale":"2023-12-11T19:33:10.341155+0000","last_undegraded":"2023-12-11T19:33:10.341155+0000","last_fullsized":"2023-12-11T19:33:10.341155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:45:08.515732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583186+0000","last_change":"2023-12-11T19:31:51.263293+0000","last_active":"2023-12-11T19:32:10.583186+0000","last_peered":"2023-12-11T19:32:10.583186+0000","last_clean":"2023-12-11T19:32:10.583186+0000","last_became_active":"2023-12-11T19:31:51.262957+0000","last_became_peered":"2023-12-11T19:31:51.262957+0000","last_unstale":"2023-12-11T19:32:10.583186+0000","last_undegraded":"2023-12-11T19:32:10.583186+0000","last_fullsized":"2023-12-11T19:32:10.583186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:21.717430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339499+0000","last_change":"2023-12-11T19:31:52.275023+0000","last_active":"2023-12-11T19:33:10.339499+0000","last_peered":"2023-12-11T19:33:10.339499+0000","last_clean":"2023-12-11T19:33:10.339499+0000","last_became_active":"2023-12-11T19:31:52.274231+0000","last_became_peered":"2023-12-11T19:31:52.274231+0000","last_unstale":"2023-12-11T19:33:10.339499+0000","last_undegraded":"2023-12-11T19:33:10.339499+0000","last_fullsized":"2023-12-11T19:33:10.339499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:01:39.529331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.583539+0000","last_change":"2023-12-11T19:31:51.256744+0000","last_active":"2023-12-11T19:32:11.583539+0000","last_peered":"2023-12-11T19:32:11.583539+0000","last_clean":"2023-12-11T19:32:11.583539+0000","last_became_active":"2023-12-11T19:31:51.256401+0000","last_became_peered":"2023-12-11T19:31:51.256401+0000","last_unstale":"2023-12-11T19:32:11.583539+0000","last_undegraded":"2023-12-11T19:32:11.583539+0000","last_fullsized":"2023-12-11T19:32:11.583539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:09:58.686350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332391+0000","last_change":"2023-12-11T19:31:52.277961+0000","last_active":"2023-12-11T19:33:10.332391+0000","last_peered":"2023-12-11T19:33:10.332391+0000","last_clean":"2023-12-11T19:33:10.332391+0000","last_became_active":"2023-12-11T19:31:52.277492+0000","last_became_peered":"2023-12-11T19:31:52.277492+0000","last_unstale":"2023-12-11T19:33:10.332391+0000","last_undegraded":"2023-12-11T19:33:10.332391+0000","last_fullsized":"2023-12-11T19:33:10.332391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:25:46.702232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.583392+0000","last_change":"2023-12-11T19:31:51.256477+0000","last_active":"2023-12-11T19:32:10.583392+0000","last_peered":"2023-12-11T19:32:10.583392+0000","last_clean":"2023-12-11T19:32:10.583392+0000","last_became_active":"2023-12-11T19:31:51.254485+0000","last_became_peered":"2023-12-11T19:31:51.254485+0000","last_unstale":"2023-12-11T19:32:10.583392+0000","last_undegraded":"2023-12-11T19:32:10.583392+0000","last_fullsized":"2023-12-11T19:32:10.583392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:23:54.918043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.324798+0000","last_change":"2023-12-11T19:31:52.271376+0000","last_active":"2023-12-11T19:33:10.324798+0000","last_peered":"2023-12-11T19:33:10.324798+0000","last_clean":"2023-12-11T19:33:10.324798+0000","last_became_active":"2023-12-11T19:31:52.269448+0000","last_became_peered":"2023-12-11T19:31:52.269448+0000","last_unstale":"2023-12-11T19:33:10.324798+0000","last_undegraded":"2023-12-11T19:33:10.324798+0000","last_fullsized":"2023-12-11T19:33:10.324798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:38:58.398246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:36.706343+0000","last_change":"2023-12-11T19:31:51.259739+0000","last_active":"2023-12-11T19:33:36.706343+0000","last_peered":"2023-12-11T19:33:36.706343+0000","last_clean":"2023-12-11T19:33:36.706343+0000","last_became_active":"2023-12-11T19:31:51.259363+0000","last_became_peered":"2023-12-11T19:31:51.259363+0000","last_unstale":"2023-12-11T19:33:36.706343+0000","last_undegraded":"2023-12-11T19:33:36.706343+0000","last_fullsized":"2023-12-11T19:33:36.706343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:11.536785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1086,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341698+0000","last_change":"2023-12-11T19:31:52.270468+0000","last_active":"2023-12-11T19:33:10.341698+0000","last_peered":"2023-12-11T19:33:10.341698+0000","last_clean":"2023-12-11T19:33:10.341698+0000","last_became_active":"2023-12-11T19:31:52.269500+0000","last_became_peered":"2023-12-11T19:31:52.269500+0000","last_unstale":"2023-12-11T19:33:10.341698+0000","last_undegraded":"2023-12-11T19:33:10.341698+0000","last_fullsized":"2023-12-11T19:33:10.341698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:51:49.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.317738+0000","last_change":"2023-12-11T19:32:11.594139+0000","last_active":"2023-12-11T19:33:10.317738+0000","last_peered":"2023-12-11T19:33:10.317738+0000","last_clean":"2023-12-11T19:33:10.317738+0000","last_became_active":"2023-12-11T19:32:11.593876+0000","last_became_peered":"2023-12-11T19:32:11.593876+0000","last_unstale":"2023-12-11T19:33:10.317738+0000","last_undegraded":"2023-12-11T19:33:10.317738+0000","last_fullsized":"2023-12-11T19:33:10.317738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:22:15.243111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":125,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.736520+0000","last_change":"2023-12-11T19:31:51.250344+0000","last_active":"2023-12-11T19:33:39.736520+0000","last_peered":"2023-12-11T19:33:39.736520+0000","last_clean":"2023-12-11T19:33:39.736520+0000","last_became_active":"2023-12-11T19:31:51.249702+0000","last_became_peered":"2023-12-11T19:31:51.249702+0000","last_unstale":"2023-12-11T19:33:39.736520+0000","last_undegraded":"2023-12-11T19:33:39.736520+0000","last_fullsized":"2023-12-11T19:33:39.736520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T07:15:11.259218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341374+0000","last_change":"2023-12-11T19:31:52.274385+0000","last_active":"2023-12-11T19:33:10.341374+0000","last_peered":"2023-12-11T19:33:10.341374+0000","last_clean":"2023-12-11T19:33:10.341374+0000","last_became_active":"2023-12-11T19:31:52.273723+0000","last_became_peered":"2023-12-11T19:31:52.273723+0000","last_unstale":"2023-12-11T19:33:10.341374+0000","last_undegraded":"2023-12-11T19:33:10.341374+0000","last_fullsized":"2023-12-11T19:33:10.341374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:01:43.419709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.582019+0000","last_change":"2023-12-11T19:31:51.254347+0000","last_active":"2023-12-11T19:32:10.582019+0000","last_peered":"2023-12-11T19:32:10.582019+0000","last_clean":"2023-12-11T19:32:10.582019+0000","last_became_active":"2023-12-11T19:31:51.253731+0000","last_became_peered":"2023-12-11T19:31:51.253731+0000","last_unstale":"2023-12-11T19:32:10.582019+0000","last_undegraded":"2023-12-11T19:32:10.582019+0000","last_fullsized":"2023-12-11T19:32:10.582019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:58:50.352080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":315,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.582905+0000","last_change":"2023-12-11T19:30:34.139123+0000","last_active":"2023-12-11T19:32:11.582905+0000","last_peered":"2023-12-11T19:32:11.582905+0000","last_clean":"2023-12-11T19:32:11.582905+0000","last_became_active":"2023-12-11T19:30:34.138556+0000","last_became_peered":"2023-12-11T19:30:34.138556+0000","last_unstale":"2023-12-11T19:32:11.582905+0000","last_undegraded":"2023-12-11T19:32:11.582905+0000","last_fullsized":"2023-12-11T19:32:11.582905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:26:55.061952+0000","last_clean_scrub_stamp":"2023-12-11T19:26:55.061952+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:22:20.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.325546+0000","last_change":"2023-12-11T19:32:11.590972+0000","last_active":"2023-12-11T19:33:10.325546+0000","last_peered":"2023-12-11T19:33:10.325546+0000","last_clean":"2023-12-11T19:33:10.325546+0000","last_became_active":"2023-12-11T19:32:11.590564+0000","last_became_peered":"2023-12-11T19:32:11.590564+0000","last_unstale":"2023-12-11T19:33:10.325546+0000","last_undegraded":"2023-12-11T19:33:10.325546+0000","last_fullsized":"2023-12-11T19:33:10.325546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:48:35.323725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.578698+0000","last_change":"2023-12-11T19:31:51.259773+0000","last_active":"2023-12-11T19:32:10.578698+0000","last_peered":"2023-12-11T19:32:10.578698+0000","last_clean":"2023-12-11T19:32:10.578698+0000","last_became_active":"2023-12-11T19:31:51.259157+0000","last_became_peered":"2023-12-11T19:31:51.259157+0000","last_unstale":"2023-12-11T19:32:10.578698+0000","last_undegraded":"2023-12-11T19:32:10.578698+0000","last_fullsized":"2023-12-11T19:32:10.578698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:13:56.172366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.584892+0000","last_change":"2023-12-11T19:31:51.265726+0000","last_active":"2023-12-11T19:32:10.584892+0000","last_peered":"2023-12-11T19:32:10.584892+0000","last_clean":"2023-12-11T19:32:10.584892+0000","last_became_active":"2023-12-11T19:31:51.264693+0000","last_became_peered":"2023-12-11T19:31:51.264693+0000","last_unstale":"2023-12-11T19:32:10.584892+0000","last_undegraded":"2023-12-11T19:32:10.584892+0000","last_fullsized":"2023-12-11T19:32:10.584892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:10:30.875780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339768+0000","last_change":"2023-12-11T19:31:52.262943+0000","last_active":"2023-12-11T19:33:10.339768+0000","last_peered":"2023-12-11T19:33:10.339768+0000","last_clean":"2023-12-11T19:33:10.339768+0000","last_became_active":"2023-12-11T19:31:52.262457+0000","last_became_peered":"2023-12-11T19:31:52.262457+0000","last_unstale":"2023-12-11T19:33:10.339768+0000","last_undegraded":"2023-12-11T19:33:10.339768+0000","last_fullsized":"2023-12-11T19:33:10.339768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:40:07.530816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.878510+0000","last_change":"2023-12-11T19:32:11.599025+0000","last_active":"2023-12-11T19:33:10.878510+0000","last_peered":"2023-12-11T19:33:10.878510+0000","last_clean":"2023-12-11T19:33:10.878510+0000","last_became_active":"2023-12-11T19:32:11.587530+0000","last_became_peered":"2023-12-11T19:32:11.587530+0000","last_unstale":"2023-12-11T19:33:10.878510+0000","last_undegraded":"2023-12-11T19:33:10.878510+0000","last_fullsized":"2023-12-11T19:33:10.878510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:08:05.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339794+0000","last_change":"2023-12-11T19:31:52.271559+0000","last_active":"2023-12-11T19:33:10.339794+0000","last_peered":"2023-12-11T19:33:10.339794+0000","last_clean":"2023-12-11T19:33:10.339794+0000","last_became_active":"2023-12-11T19:31:52.271188+0000","last_became_peered":"2023-12-11T19:31:52.271188+0000","last_unstale":"2023-12-11T19:33:10.339794+0000","last_undegraded":"2023-12-11T19:33:10.339794+0000","last_fullsized":"2023-12-11T19:33:10.339794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:57.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581975+0000","last_change":"2023-12-11T19:31:51.256244+0000","last_active":"2023-12-11T19:32:10.581975+0000","last_peered":"2023-12-11T19:32:10.581975+0000","last_clean":"2023-12-11T19:32:10.581975+0000","last_became_active":"2023-12-11T19:31:51.253545+0000","last_became_peered":"2023-12-11T19:31:51.253545+0000","last_unstale":"2023-12-11T19:32:10.581975+0000","last_undegraded":"2023-12-11T19:32:10.581975+0000","last_fullsized":"2023-12-11T19:32:10.581975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:31:00.500964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342950+0000","last_change":"2023-12-11T19:31:52.269401+0000","last_active":"2023-12-11T19:33:10.342950+0000","last_peered":"2023-12-11T19:33:10.342950+0000","last_clean":"2023-12-11T19:33:10.342950+0000","last_became_active":"2023-12-11T19:31:52.269010+0000","last_became_peered":"2023-12-11T19:31:52.269010+0000","last_unstale":"2023-12-11T19:33:10.342950+0000","last_undegraded":"2023-12-11T19:33:10.342950+0000","last_fullsized":"2023-12-11T19:33:10.342950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:02:28.334806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581192+0000","last_change":"2023-12-11T19:31:51.252066+0000","last_active":"2023-12-11T19:32:10.581192+0000","last_peered":"2023-12-11T19:32:10.581192+0000","last_clean":"2023-12-11T19:32:10.581192+0000","last_became_active":"2023-12-11T19:31:51.251556+0000","last_became_peered":"2023-12-11T19:31:51.251556+0000","last_unstale":"2023-12-11T19:32:10.581192+0000","last_undegraded":"2023-12-11T19:32:10.581192+0000","last_fullsized":"2023-12-11T19:32:10.581192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:05:48.107948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.287117+0000","last_change":"2023-12-11T19:31:52.266311+0000","last_active":"2023-12-11T19:33:10.287117+0000","last_peered":"2023-12-11T19:33:10.287117+0000","last_clean":"2023-12-11T19:33:10.287117+0000","last_became_active":"2023-12-11T19:31:52.265964+0000","last_became_peered":"2023-12-11T19:31:52.265964+0000","last_unstale":"2023-12-11T19:33:10.287117+0000","last_undegraded":"2023-12-11T19:33:10.287117+0000","last_fullsized":"2023-12-11T19:33:10.287117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:34:02.292298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.585191+0000","last_change":"2023-12-11T19:31:51.264527+0000","last_active":"2023-12-11T19:32:10.585191+0000","last_peered":"2023-12-11T19:32:10.585191+0000","last_clean":"2023-12-11T19:32:10.585191+0000","last_became_active":"2023-12-11T19:31:51.257093+0000","last_became_peered":"2023-12-11T19:31:51.257093+0000","last_unstale":"2023-12-11T19:32:10.585191+0000","last_undegraded":"2023-12-11T19:32:10.585191+0000","last_fullsized":"2023-12-11T19:32:10.585191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:36:30.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339172+0000","last_change":"2023-12-11T19:31:52.269937+0000","last_active":"2023-12-11T19:33:10.339172+0000","last_peered":"2023-12-11T19:33:10.339172+0000","last_clean":"2023-12-11T19:33:10.339172+0000","last_became_active":"2023-12-11T19:31:52.268989+0000","last_became_peered":"2023-12-11T19:31:52.268989+0000","last_unstale":"2023-12-11T19:33:10.339172+0000","last_undegraded":"2023-12-11T19:33:10.339172+0000","last_fullsized":"2023-12-11T19:33:10.339172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:40:49.215432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.588890+0000","last_change":"2023-12-11T19:31:51.258238+0000","last_active":"2023-12-11T19:32:11.588890+0000","last_peered":"2023-12-11T19:32:11.588890+0000","last_clean":"2023-12-11T19:32:11.588890+0000","last_became_active":"2023-12-11T19:31:51.257840+0000","last_became_peered":"2023-12-11T19:31:51.257840+0000","last_unstale":"2023-12-11T19:32:11.588890+0000","last_undegraded":"2023-12-11T19:32:11.588890+0000","last_fullsized":"2023-12-11T19:32:11.588890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:27:36.416399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.342653+0000","last_change":"2023-12-11T19:31:52.279124+0000","last_active":"2023-12-11T19:33:10.342653+0000","last_peered":"2023-12-11T19:33:10.342653+0000","last_clean":"2023-12-11T19:33:10.342653+0000","last_became_active":"2023-12-11T19:31:52.278805+0000","last_became_peered":"2023-12-11T19:31:52.278805+0000","last_unstale":"2023-12-11T19:33:10.342653+0000","last_undegraded":"2023-12-11T19:33:10.342653+0000","last_fullsized":"2023-12-11T19:33:10.342653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T06:27:45.595124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:32:11.599949+0000","last_change":"2023-12-11T19:32:11.599116+0000","last_active":"2023-12-11T19:32:11.599949+0000","last_peered":"2023-12-11T19:32:11.599949+0000","last_clean":"2023-12-11T19:32:11.599949+0000","last_became_active":"2023-12-11T19:32:11.592106+0000","last_became_peered":"2023-12-11T19:32:11.592106+0000","last_unstale":"2023-12-11T19:32:11.599949+0000","last_undegraded":"2023-12-11T19:32:11.599949+0000","last_fullsized":"2023-12-11T19:32:11.599949+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:23.285224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.344074+0000","last_change":"2023-12-11T19:31:52.278149+0000","last_active":"2023-12-11T19:33:10.344074+0000","last_peered":"2023-12-11T19:33:10.344074+0000","last_clean":"2023-12-11T19:33:10.344074+0000","last_became_active":"2023-12-11T19:31:52.277818+0000","last_became_peered":"2023-12-11T19:31:52.277818+0000","last_unstale":"2023-12-11T19:33:10.344074+0000","last_undegraded":"2023-12-11T19:33:10.344074+0000","last_fullsized":"2023-12-11T19:33:10.344074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:43:25.888444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:39.732938+0000","last_change":"2023-12-11T19:32:11.603249+0000","last_active":"2023-12-11T19:33:39.732938+0000","last_peered":"2023-12-11T19:33:39.732938+0000","last_clean":"2023-12-11T19:33:39.732938+0000","last_became_active":"2023-12-11T19:32:11.587129+0000","last_became_peered":"2023-12-11T19:32:11.587129+0000","last_unstale":"2023-12-11T19:33:39.732938+0000","last_undegraded":"2023-12-11T19:33:39.732938+0000","last_fullsized":"2023-12-11T19:33:39.732938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:28:44.762360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1668,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343509+0000","last_change":"2023-12-11T19:31:52.275477+0000","last_active":"2023-12-11T19:33:10.343509+0000","last_peered":"2023-12-11T19:33:10.343509+0000","last_clean":"2023-12-11T19:33:10.343509+0000","last_became_active":"2023-12-11T19:31:52.273025+0000","last_became_peered":"2023-12-11T19:31:52.273025+0000","last_unstale":"2023-12-11T19:33:10.343509+0000","last_undegraded":"2023-12-11T19:33:10.343509+0000","last_fullsized":"2023-12-11T19:33:10.343509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:33:59.499162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581433+0000","last_change":"2023-12-11T19:31:51.264436+0000","last_active":"2023-12-11T19:32:10.581433+0000","last_peered":"2023-12-11T19:32:10.581433+0000","last_clean":"2023-12-11T19:32:10.581433+0000","last_became_active":"2023-12-11T19:31:51.263490+0000","last_became_peered":"2023-12-11T19:31:51.263490+0000","last_unstale":"2023-12-11T19:32:10.581433+0000","last_undegraded":"2023-12-11T19:32:10.581433+0000","last_fullsized":"2023-12-11T19:32:10.581433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:52:47.520776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333570+0000","last_change":"2023-12-11T19:32:11.596576+0000","last_active":"2023-12-11T19:33:10.333570+0000","last_peered":"2023-12-11T19:33:10.333570+0000","last_clean":"2023-12-11T19:33:10.333570+0000","last_became_active":"2023-12-11T19:32:11.592679+0000","last_became_peered":"2023-12-11T19:32:11.592679+0000","last_unstale":"2023-12-11T19:33:10.333570+0000","last_undegraded":"2023-12-11T19:33:10.333570+0000","last_fullsized":"2023-12-11T19:33:10.333570+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:26:20.450580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T19:32:10.581645+0000","last_change":"2023-12-11T19:31:51.250269+0000","last_active":"2023-12-11T19:32:10.581645+0000","last_peered":"2023-12-11T19:32:10.581645+0000","last_clean":"2023-12-11T19:32:10.581645+0000","last_became_active":"2023-12-11T19:31:51.249363+0000","last_became_peered":"2023-12-11T19:31:51.249363+0000","last_unstale":"2023-12-11T19:32:10.581645+0000","last_undegraded":"2023-12-11T19:32:10.581645+0000","last_fullsized":"2023-12-11T19:32:10.581645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:50.029380+0000","last_clean_scrub_stamp":"2023-12-11T19:31:50.029380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:18:40.616411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331611+0000","last_change":"2023-12-11T19:31:52.271322+0000","last_active":"2023-12-11T19:33:10.331611+0000","last_peered":"2023-12-11T19:33:10.331611+0000","last_clean":"2023-12-11T19:33:10.331611+0000","last_became_active":"2023-12-11T19:31:52.271005+0000","last_became_peered":"2023-12-11T19:31:52.271005+0000","last_unstale":"2023-12-11T19:33:10.331611+0000","last_undegraded":"2023-12-11T19:33:10.331611+0000","last_fullsized":"2023-12-11T19:33:10.331611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:35:50.101445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.335239+0000","last_change":"2023-12-11T19:31:52.270608+0000","last_active":"2023-12-11T19:33:10.335239+0000","last_peered":"2023-12-11T19:33:10.335239+0000","last_clean":"2023-12-11T19:33:10.335239+0000","last_became_active":"2023-12-11T19:31:52.270237+0000","last_became_peered":"2023-12-11T19:31:52.270237+0000","last_unstale":"2023-12-11T19:33:10.335239+0000","last_undegraded":"2023-12-11T19:33:10.335239+0000","last_fullsized":"2023-12-11T19:33:10.335239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:49.260219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333471+0000","last_change":"2023-12-11T19:31:52.268607+0000","last_active":"2023-12-11T19:33:10.333471+0000","last_peered":"2023-12-11T19:33:10.333471+0000","last_clean":"2023-12-11T19:33:10.333471+0000","last_became_active":"2023-12-11T19:31:52.268301+0000","last_became_peered":"2023-12-11T19:31:52.268301+0000","last_unstale":"2023-12-11T19:33:10.333471+0000","last_undegraded":"2023-12-11T19:33:10.333471+0000","last_fullsized":"2023-12-11T19:33:10.333471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:42:17.237540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.332291+0000","last_change":"2023-12-11T19:31:52.273868+0000","last_active":"2023-12-11T19:33:10.332291+0000","last_peered":"2023-12-11T19:33:10.332291+0000","last_clean":"2023-12-11T19:33:10.332291+0000","last_became_active":"2023-12-11T19:31:52.272148+0000","last_became_peered":"2023-12-11T19:31:52.272148+0000","last_unstale":"2023-12-11T19:33:10.332291+0000","last_undegraded":"2023-12-11T19:33:10.332291+0000","last_fullsized":"2023-12-11T19:33:10.332291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:22:53.518347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.336523+0000","last_change":"2023-12-11T19:32:11.586739+0000","last_active":"2023-12-11T19:33:10.336523+0000","last_peered":"2023-12-11T19:33:10.336523+0000","last_clean":"2023-12-11T19:33:10.336523+0000","last_became_active":"2023-12-11T19:32:11.586457+0000","last_became_peered":"2023-12-11T19:32:11.586457+0000","last_unstale":"2023-12-11T19:33:10.336523+0000","last_undegraded":"2023-12-11T19:33:10.336523+0000","last_fullsized":"2023-12-11T19:33:10.336523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:50.986234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341946+0000","last_change":"2023-12-11T19:31:52.277324+0000","last_active":"2023-12-11T19:33:10.341946+0000","last_peered":"2023-12-11T19:33:10.341946+0000","last_clean":"2023-12-11T19:33:10.341946+0000","last_became_active":"2023-12-11T19:31:52.276927+0000","last_became_peered":"2023-12-11T19:31:52.276927+0000","last_unstale":"2023-12-11T19:33:10.341946+0000","last_undegraded":"2023-12-11T19:33:10.341946+0000","last_fullsized":"2023-12-11T19:33:10.341946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:46:21.047790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.333225+0000","last_change":"2023-12-11T19:31:52.268543+0000","last_active":"2023-12-11T19:33:10.333225+0000","last_peered":"2023-12-11T19:33:10.333225+0000","last_clean":"2023-12-11T19:33:10.333225+0000","last_became_active":"2023-12-11T19:31:52.268143+0000","last_became_peered":"2023-12-11T19:31:52.268143+0000","last_unstale":"2023-12-11T19:33:10.333225+0000","last_undegraded":"2023-12-11T19:33:10.333225+0000","last_fullsized":"2023-12-11T19:33:10.333225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:42:24.043020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.339819+0000","last_change":"2023-12-11T19:31:52.264092+0000","last_active":"2023-12-11T19:33:10.339819+0000","last_peered":"2023-12-11T19:33:10.339819+0000","last_clean":"2023-12-11T19:33:10.339819+0000","last_became_active":"2023-12-11T19:31:52.263834+0000","last_became_peered":"2023-12-11T19:31:52.263834+0000","last_unstale":"2023-12-11T19:33:10.339819+0000","last_undegraded":"2023-12-11T19:33:10.339819+0000","last_fullsized":"2023-12-11T19:33:10.339819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:00:35.759318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.319673+0000","last_change":"2023-12-11T19:31:52.277429+0000","last_active":"2023-12-11T19:33:10.319673+0000","last_peered":"2023-12-11T19:33:10.319673+0000","last_clean":"2023-12-11T19:33:10.319673+0000","last_became_active":"2023-12-11T19:31:52.277082+0000","last_became_peered":"2023-12-11T19:31:52.277082+0000","last_unstale":"2023-12-11T19:33:10.319673+0000","last_undegraded":"2023-12-11T19:33:10.319673+0000","last_fullsized":"2023-12-11T19:33:10.319673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:34.639454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.331108+0000","last_change":"2023-12-11T19:31:52.274035+0000","last_active":"2023-12-11T19:33:10.331108+0000","last_peered":"2023-12-11T19:33:10.331108+0000","last_clean":"2023-12-11T19:33:10.331108+0000","last_became_active":"2023-12-11T19:31:52.272734+0000","last_became_peered":"2023-12-11T19:31:52.272734+0000","last_unstale":"2023-12-11T19:33:10.331108+0000","last_undegraded":"2023-12-11T19:33:10.331108+0000","last_fullsized":"2023-12-11T19:33:10.331108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:12:27.871779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337463+0000","last_change":"2023-12-11T19:31:52.276143+0000","last_active":"2023-12-11T19:33:10.337463+0000","last_peered":"2023-12-11T19:33:10.337463+0000","last_clean":"2023-12-11T19:33:10.337463+0000","last_became_active":"2023-12-11T19:31:52.275822+0000","last_became_peered":"2023-12-11T19:31:52.275822+0000","last_unstale":"2023-12-11T19:33:10.337463+0000","last_undegraded":"2023-12-11T19:33:10.337463+0000","last_fullsized":"2023-12-11T19:33:10.337463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:52:10.500076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.343501+0000","last_change":"2023-12-11T19:31:52.274164+0000","last_active":"2023-12-11T19:33:10.343501+0000","last_peered":"2023-12-11T19:33:10.343501+0000","last_clean":"2023-12-11T19:33:10.343501+0000","last_became_active":"2023-12-11T19:31:52.271647+0000","last_became_peered":"2023-12-11T19:31:52.271647+0000","last_unstale":"2023-12-11T19:33:10.343501+0000","last_undegraded":"2023-12-11T19:33:10.343501+0000","last_fullsized":"2023-12-11T19:33:10.343501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:32:04.306851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.341666+0000","last_change":"2023-12-11T19:31:52.268896+0000","last_active":"2023-12-11T19:33:10.341666+0000","last_peered":"2023-12-11T19:33:10.341666+0000","last_clean":"2023-12-11T19:33:10.341666+0000","last_became_active":"2023-12-11T19:31:52.267080+0000","last_became_peered":"2023-12-11T19:31:52.267080+0000","last_unstale":"2023-12-11T19:33:10.341666+0000","last_undegraded":"2023-12-11T19:33:10.341666+0000","last_fullsized":"2023-12-11T19:33:10.341666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:21:14.257200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T19:33:10.337779+0000","last_change":"2023-12-11T19:31:52.271033+0000","last_active":"2023-12-11T19:33:10.337779+0000","last_peered":"2023-12-11T19:33:10.337779+0000","last_clean":"2023-12-11T19:33:10.337779+0000","last_became_active":"2023-12-11T19:31:52.270698+0000","last_became_peered":"2023-12-11T19:31:52.270698+0000","last_unstale":"2023-12-11T19:33:10.337779+0000","last_undegraded":"2023-12-11T19:33:10.337779+0000","last_fullsized":"2023-12-11T19:33:10.337779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T19:31:51.228240+0000","last_clean_scrub_stamp":"2023-12-11T19:31:51.228240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:57:30.426299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284701,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70230,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1045,"num_read_kb":608,"num_write":255,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2181,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":210690,"data_compressed":5496,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":460267,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.94399999999999995,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80300000000000005,"5min":1.454,"15min":1.454},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95299999999999996,"5min":1.331,"15min":1.331},"last":0.83599999999999997}]},{"osd":2,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.65800000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.077,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.56799999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.96599999999999997,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93400000000000005,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.60399999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.052,"5min":1.518,"15min":1.518},"last":0.80700000000000005}]},{"osd":6,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.85099999999999998,"15min":0.871},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0720000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.85499999999999998,"15min":0.873},"min":{"1min":0.58399999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0980000000000001,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.872,"15min":0.86199999999999999},"min":{"1min":0.77900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.115,"5min":1.403,"15min":1.403},"last":1},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89500000000000002,"15min":0.90100000000000002},"min":{"1min":0.64200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.038,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88400000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.86199999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.69899999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1419999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.79200000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.165,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Mon Dec 11 19:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.98299999999999998},"min":{"1min":0.67500000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0629999999999999,"5min":1.49,"15min":1.49},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97999999999999998},"min":{"1min":0.76100000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1819999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.90000000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":1112588,"data_compressed":43838,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.8120000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.36899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.376,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91000000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002}]},{"osd":2,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.57599999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.96099999999999997,"5min":1.04,"15min":1.04},"last":0.87},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73199999999999998,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.026,"5min":1.032,"15min":1.032},"last":0.93700000000000006}]},{"osd":3,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66800000000000004,"15min":0.67900000000000005},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.92600000000000005,"5min":1.24,"15min":1.24},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95299999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.66600000000000004,"15min":0.70199999999999996},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76700000000000002}]},{"osd":5,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.81100000000000005},"min":{"1min":0.61699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98399999999999999,"5min":1.085,"15min":1.085},"last":0.90600000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.58999999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.97099999999999997,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.006,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Dec 11 19:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.443}]},{"osd":8,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89800000000000002,"15min":0.90400000000000003},"min":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0800000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.89400000000000002,"15min":0.92800000000000005},"min":{"1min":0.627,"5min":0.626,"15min":0.626},"max":{"1min":0.995,"5min":1.643,"15min":1.643},"last":1.026}]},{"osd":9,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89900000000000002,"15min":0.90700000000000003},"min":{"1min":0.69499999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0629999999999999,"5min":1.321,"15min":1.321},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.871,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.98899999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.97499999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.871,"15min":0.873},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1259999999999999,"5min":1.224,"15min":1.224},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.90700000000000003,"15min":0.90200000000000002},"min":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1379999999999999,"5min":1.161,"15min":1.161},"last":0.99099999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":459822,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68899999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.101,"5min":6.101,"15min":6.101},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.74399999999999999,"15min":0.76700000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.85199999999999998,"15min":0.84399999999999997},"min":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.72699999999999998,"15min":0.747},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0589999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":6.0510000000000002,"5min":6.0510000000000002,"15min":6.0510000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.82899999999999996,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.024,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.875,"15min":0.91400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.278,"15min":1.278},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88,"15min":0.90800000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Dec 11 19:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.88100000000000001,"15min":0.89800000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0429999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90500000000000003,"15min":0.93200000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0640000000000001,"5min":1.389,"15min":1.389},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.92200000000000004,"15min":0.96799999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0129999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.94699999999999995,"15min":0.97699999999999998},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.081,"5min":1.274,"15min":1.274},"last":0.79900000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.95599999999999996,"15min":1.0029999999999999},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97499999999999998,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.97699999999999998,"15min":1.014},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.0309999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:33:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.93899999999999995,"15min":0.99099999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.98799999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.0089999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.85399999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.61799999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":460077,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":1.097,"15min":1.097},"last":0.60999999999999999}]},{"osd":1,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.55700000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1479999999999999,"5min":1.351,"15min":1.351},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.81200000000000006,"15min":0.80900000000000005},"min":{"1min":0.61399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.0740000000000001,"5min":2.21,"15min":2.21},"last":0.86699999999999999}]},{"osd":2,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.92900000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.62,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.292,"15min":1.292},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.2350000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.876}]},{"osd":5,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.58199999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1140000000000001,"5min":2.173,"15min":2.173},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.083,"5min":1.177,"15min":1.177},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.84899999999999998,"15min":0.82099999999999995},"min":{"1min":0.74199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2869999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.022},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1459999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.71699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.304,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.74099999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85199999999999998}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.86599999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.256,"5min":2.4470000000000001,"15min":2.4470000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.01,"15min":0.97499999999999998},"min":{"1min":0.94099999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.3600000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.113}]},{"osd":10,"last update":"Mon Dec 11 19:33:39 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.94299999999999995,"15min":0.89300000000000002},"min":{"1min":0.68400000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.246,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.103},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0169999999999999,"15min":0.98199999999999998},"min":{"1min":0.88100000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.3460000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.98299999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:32:53 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.1319999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1054433,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:31:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.51600000000000001,"15min":0.53600000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76900000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.5},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59599999999999997},"min":{"1min":0.35399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.165},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77700000000000002,"15min":0.77000000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.89000000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.754,"15min":0.80600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97199999999999998,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70399999999999996,"15min":0.73799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59299999999999997,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73199999999999998,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.042,"5min":1.238,"15min":1.238},"last":0.626}]},{"osd":5,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96899999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.026,"5min":1.28,"15min":1.28},"last":0.998}]},{"osd":6,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79900000000000004,"15min":0.85499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.512,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.751,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.83699999999999997,"5min":1.359,"15min":1.359},"last":0.68000000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97999999999999998,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86599999999999999,"15min":0.85999999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97099999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.872}]},{"osd":9,"last update":"Mon Dec 11 19:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.879,"15min":0.88800000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.98399999999999999,"5min":1.296,"15min":1.296},"last":0.753}]},{"osd":10,"last update":"Mon Dec 11 19:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.90700000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0369999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.90500000000000003,"15min":0.91800000000000004},"min":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.194,"5min":1.25,"15min":1.25},"last":0.77700000000000002}]},{"osd":11,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.028},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.94499999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459920,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48299999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85999999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.046,"5min":1.089,"15min":1.089},"last":0.80700000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73099999999999998,"15min":0.70499999999999996},"min":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0269999999999999,"5min":1.284,"15min":1.284},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.494,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60599999999999998}]},{"osd":3,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.40200000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80000000000000004},"min":{"1min":0.58099999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.0860000000000001,"5min":1.157,"15min":1.157},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.84099999999999997,"15min":0.81799999999999995},"min":{"1min":0.58999999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86899999999999999,"15min":0.91600000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.603,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88500000000000001,"15min":0.87},"min":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.137,"5min":1.234,"15min":1.234},"last":0.93100000000000005}]},{"osd":8,"last update":"Mon Dec 11 19:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":0.99399999999999999,"5min":1.327,"15min":1.327},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.871,"15min":0.88100000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.375,"5min":1.379,"15min":1.379},"last":0.79100000000000004}]},{"osd":9,"last update":"Mon Dec 11 19:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.90900000000000003,"15min":0.91700000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3220000000000001,"5min":1.345,"15min":1.345},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.876}]},{"osd":10,"last update":"Mon Dec 11 19:33:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83599999999999997,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.94699999999999995,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.96699999999999997,"15min":0.97099999999999997},"min":{"1min":0.70999999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.081,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522978,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.55700000000000005},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69099999999999995,"15min":0.58199999999999996},"min":{"1min":0.58099999999999996,"5min":0.377,"15min":0.33600000000000002},"max":{"1min":1.05,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.79900000000000004,"15min":0.72099999999999997},"min":{"1min":0.67600000000000005,"5min":0.42399999999999999,"15min":0.38400000000000001},"max":{"1min":1.1399999999999999,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.752,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.435,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.99299999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.81100000000000005,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":14.044,"5min":14.044,"15min":14.044},"last":0.83099999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80600000000000005,"5min":1.022,"15min":1.022},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.66600000000000004,"15min":0.57199999999999995},"min":{"1min":0.61299999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.083,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Mon Dec 11 19:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.749,"15min":0.68100000000000005},"min":{"1min":0.57299999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1539999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.73599999999999999,"15min":0.85599999999999998},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93600000000000005,"5min":5.5030000000000001,"15min":5.5030000000000001},"last":1.01}]},{"osd":6,"last update":"Mon Dec 11 19:32:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.77800000000000002,"15min":0.72999999999999998},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.627,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.997}]},{"osd":7,"last update":"Mon Dec 11 19:33:21 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.74199999999999999,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.68000000000000005,"15min":0.627},"min":{"1min":0.501,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79300000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:32:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.68700000000000006,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2110000000000001,"5min":1.52,"15min":1.52},"last":1.127},{"interface":"front","average":{"1min":1.091,"5min":0.88500000000000001,"15min":0.83799999999999997},"min":{"1min":0.67100000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.1419999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:10 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.91700000000000004,"15min":0.875},"min":{"1min":0.77400000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.343,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.008,"5min":0.91800000000000004,"15min":0.88400000000000001},"min":{"1min":0.81200000000000006,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.105}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.127,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.008,"5min":0.96499999999999997,"15min":0.93899999999999995},"min":{"1min":0.79700000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.155}]},{"osd":11,"last update":"Mon Dec 11 19:32:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0509999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":461253,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.82799999999999996,"15min":0.67800000000000005},"min":{"1min":0.52300000000000002,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":1.1890000000000001,"5min":7.3300000000000001,"15min":7.3300000000000001},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.34200000000000003,"15min":0.33600000000000002},"max":{"1min":1.0449999999999999,"5min":7.2629999999999999,"15min":7.2629999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.748,"15min":0.64600000000000002},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0189999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.41399999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.403},"last":0.63500000000000001}]},{"osd":3,"last update":"Mon Dec 11 19:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Mon Dec 11 19:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71899999999999997,"15min":0.67800000000000005},"min":{"1min":0.55100000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:33:17 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.64900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.3799999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79400000000000004,"15min":0.81799999999999995},"min":{"1min":0.58599999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.4099999999999999,"5min":1.819,"15min":1.819},"last":1.0209999999999999}]},{"osd":6,"last update":"Mon Dec 11 19:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.081,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.038},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.80700000000000005}]},{"osd":7,"last update":"Mon Dec 11 19:31:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.56599999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.88900000000000001,"15min":0.88300000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79500000000000004}]},{"osd":8,"last update":"Mon Dec 11 19:32:43 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89900000000000002},"min":{"1min":0.63400000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.90400000000000003,"15min":0.89500000000000002},"min":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1579999999999999,"5min":1.25,"15min":1.25},"last":0.98799999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.96399999999999997,"15min":1.0069999999999999},"min":{"1min":0.57099999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92000000000000004,"15min":0.95899999999999996},"min":{"1min":0.58699999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.85199999999999998}]},{"osd":10,"last update":"Mon Dec 11 19:32:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.085},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96499999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.88900000000000001}]},{"osd":11,"last update":"Mon Dec 11 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.90100000000000002,"15min":0.88100000000000001},"min":{"1min":0.82099999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1850000000000001,"5min":0.95799999999999996,"15min":0.92000000000000004},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":6.9299999999999997,"5min":6.9299999999999997,"15min":6.9299999999999997},"last":1.103}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":519513,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.44,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.753,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.95199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71599999999999997,"15min":0.64800000000000002},"min":{"1min":0.48699999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":1.0760000000000001,"5min":1.115,"15min":1.115},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.64800000000000002},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.873}]},{"osd":3,"last update":"Mon Dec 11 19:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63700000000000001,"15min":0.60399999999999998},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":1.056,"15min":1.056},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.65200000000000002},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.03,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Mon Dec 11 19:32:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.93799999999999994}]},{"osd":5,"last update":"Mon Dec 11 19:33:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.372,"15min":0.372},"max":{"1min":0.97399999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86499999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Dec 11 19:32:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.747,"15min":0.70099999999999996},"min":{"1min":0.61799999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Mon Dec 11 19:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Dec 11 19:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.76800000000000002,"15min":0.747},"min":{"1min":0.64000000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0660000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.751},"min":{"1min":0.59499999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.119,"5min":1.133,"15min":1.133},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.86099999999999999,"15min":0.83299999999999996},"min":{"1min":0.70299999999999996,"5min":0.627,"15min":0.627},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.86599999999999999},"min":{"1min":0.68000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.137,"5min":1.167,"15min":1.167},"last":0.92100000000000004}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.91400000000000003,"15min":0.89500000000000002},"min":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.095,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91500000000000004,"15min":0.88900000000000001},"min":{"1min":0.69299999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.109,"5min":1.169,"15min":1.169},"last":0.97799999999999998}]},{"osd":11,"last update":"Mon Dec 11 19:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85399999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":458799,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48499999999999999,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.29699999999999999,"15min":0.249},"max":{"1min":0.80700000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.28599999999999998},"max":{"1min":0.93400000000000005,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89400000000000002,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71399999999999997,"15min":0.625},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98999999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.93999999999999995}]},{"osd":2,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.64700000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.33100000000000002},"max":{"1min":0.96999999999999997,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.65100000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.36199999999999999},"max":{"1min":1.0660000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88}]},{"osd":4,"last update":"Mon Dec 11 19:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.751,"15min":0.66700000000000004},"min":{"1min":0.57599999999999996,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.80500000000000005,"15min":0.71699999999999997},"min":{"1min":0.65300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.91000000000000003}]},{"osd":5,"last update":"Mon Dec 11 19:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.198,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77000000000000002,"15min":0.74199999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1220000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.006}]},{"osd":6,"last update":"Mon Dec 11 19:32:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74199999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.196,"5min":1.28,"15min":1.28},"last":0.746}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86499999999999999,"15min":0.82699999999999996},"min":{"1min":0.68600000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1399999999999999,"5min":1.752,"15min":1.752},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86499999999999999,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.206,"5min":1.631,"15min":1.631},"last":1.0620000000000001}]},{"osd":8,"last update":"Mon Dec 11 19:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.89800000000000002,"15min":0.88200000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0169999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:08 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93899999999999995,"15min":0.96399999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.042},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.95899999999999996,"15min":1.0029999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3220000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Dec 11 19:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.95399999999999996,"15min":0.97299999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.98499999999999999,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.111},{"interface":"front","average":{"1min":0.88700000000000001,"5min":1.0189999999999999,"15min":1.0549999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1000000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.095}]},{"osd":11,"last update":"Mon Dec 11 19:32:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.141},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0269999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461955,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.77300000000000002}]},{"osd":1,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.496},"min":{"1min":0.39400000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85199999999999998,"5min":1.294,"15min":1.294},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.69299999999999995,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90500000000000003,"5min":1.018,"15min":1.018},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.75,"15min":0.72899999999999998},"min":{"1min":0.58899999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0209999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.98699999999999999}]},{"osd":3,"last update":"Mon Dec 11 19:33:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69999999999999996,"15min":0.65900000000000003},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0680000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.877},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.76900000000000002,"15min":0.70799999999999996},"min":{"1min":0.57799999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0620000000000001,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.73499999999999999,"15min":0.71999999999999997},"min":{"1min":0.51800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0069999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Mon Dec 11 19:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.72999999999999998,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.395,"5min":1.429,"15min":1.429},"last":0.94199999999999995}]},{"osd":7,"last update":"Mon Dec 11 19:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82599999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72799999999999998,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.997,"5min":1.26,"15min":1.26},"last":0.96199999999999997}]},{"osd":8,"last update":"Mon Dec 11 19:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.67600000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82499999999999996,"15min":0.79400000000000004},"min":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0369999999999999}]},{"osd":9,"last update":"Mon Dec 11 19:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.109,"5min":1.494,"15min":1.494},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.81799999999999995,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.76600000000000001}]},{"osd":10,"last update":"Mon Dec 11 19:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.89700000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1100000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94399999999999995,"15min":0.95499999999999996},"min":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0960000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Mon Dec 11 19:32:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.999}]}]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1052916,"data_compressed":42006,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.624},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83799999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60999999999999999,"15min":0.65200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83899999999999997,"5min":1.177,"15min":1.177},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69599999999999995,"15min":0.66600000000000004},"min":{"1min":0.53000000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.71999999999999997,"15min":0.69199999999999995},"min":{"1min":0.56299999999999994,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98999999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.47099999999999997},"min":{"1min":0.372,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.879,"15min":0.879},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94499999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52400000000000002}]},{"osd":3,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.74199999999999999},"min":{"1min":0.56599999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96699999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.83399999999999996}]},{"osd":4,"last update":"Mon Dec 11 19:33:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96499999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.74099999999999999},"min":{"1min":0.51900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91000000000000003,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Mon Dec 11 19:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.58799999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.748,"15min":0.745},"min":{"1min":0.55500000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1240000000000001,"5min":1.597,"15min":1.597},"last":0.66500000000000004}]},{"osd":7,"last update":"Mon Dec 11 19:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78000000000000003,"15min":0.75},"min":{"1min":0.58799999999999997,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.751},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.83999999999999997,"15min":0.80900000000000005},"min":{"1min":0.71599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0760000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Mon Dec 11 19:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77300000000000002,"15min":0.75600000000000001},"min":{"1min":0.45600000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.214,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.873},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83299999999999996},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.77200000000000002}]},{"osd":9,"last update":"Mon Dec 11 19:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83299999999999996,"15min":0.83899999999999997},"min":{"1min":0.64900000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.88100000000000001,"15min":0.88800000000000001},"min":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.2509999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Mon Dec 11 19:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91500000000000004,"15min":0.90900000000000003},"min":{"1min":0.81399999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1060000000000001,"5min":1.548,"15min":1.548},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.002,"5min":0.94499999999999995,"15min":0.93100000000000005},"min":{"1min":0.77200000000000002,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34562,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36203,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64674,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60936,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63520,"data_compressed":1832,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T19:33:43.050 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:43.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:42 smithi070 ceph-mon[104044]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:43.145 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:42 smithi066 ceph-mon[97920]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:43.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:42 smithi167 ceph-mon[102750]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 -- 172.21.15.66:0/3625543297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69701014b0 msgr2=0x7f69701018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 --2- 172.21.15.66:0/3625543297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69701014b0 0x7f69701018d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f69600099c0 tx=0x7f6960009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 -- 172.21.15.66:0/3625543297 shutdown_connections 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 --2- 172.21.15.66:0/3625543297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6970103040 0x7f69701098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 --2- 172.21.15.66:0/3625543297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69701014b0 0x7f69701018d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 --2- 172.21.15.66:0/3625543297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69701026a0 0x7f6970102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 -- 172.21.15.66:0/3625543297 >> 172.21.15.66:0/3625543297 conn(0x7f69700fca20 msgr2=0x7f69700fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 -- 172.21.15.66:0/3625543297 shutdown_connections 2023-12-11T19:33:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.296+0000 7f69755ff700 1 -- 172.21.15.66:0/3625543297 wait complete. 2023-12-11T19:33:43.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.297+0000 7f69755ff700 1 Processor -- start 2023-12-11T19:33:43.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.297+0000 7f69755ff700 1 -- start start 2023-12-11T19:33:43.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f69755ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69701014b0 0x7f69701973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f69755ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69701026a0 0x7f6970197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f69755ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 0x7f697019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f69755ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697019eeb0 con 0x7f6970103040 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f69755ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697019eff0 con 0x7f69701026a0 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f69755ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697019f180 con 0x7f69701014b0 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f696f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 0x7f697019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f696f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 0x7f697019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50810/0 (socket says 172.21.15.66:50810) 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f696f7fe700 1 -- 172.21.15.66:0/1610761995 learned_addr learned my addr 172.21.15.66:0/1610761995 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f696effd700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69701014b0 0x7f69701973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.298+0000 7f6967fff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69701026a0 0x7f6970197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696f7fe700 1 -- 172.21.15.66:0/1610761995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69701014b0 msgr2=0x7f69701973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696f7fe700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69701014b0 0x7f69701973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696f7fe700 1 -- 172.21.15.66:0/1610761995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69701026a0 msgr2=0x7f6970197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696f7fe700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69701026a0 0x7f6970197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696f7fe700 1 -- 172.21.15.66:0/1610761995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600095f0 con 0x7f6970103040 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f6967fff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69701026a0 0x7f6970197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696f7fe700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 0x7f697019e8e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f696800ef30 tx=0x7f696800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968018070 con 0x7f6970103040 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f696800f040 con 0x7f6970103040 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697019f400 con 0x7f6970103040 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800cc70 con 0x7f6970103040 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.299+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697019f8c0 con 0x7f6970103040 2023-12-11T19:33:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.300+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69701089c0 con 0x7f6970103040 2023-12-11T19:33:43.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.302+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6968007500 con 0x7f6970103040 2023-12-11T19:33:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.304+0000 7f696cff9700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f695c0a1480 0x7f695c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.304+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f69680cc800 con 0x7f6970103040 2023-12-11T19:33:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.304+0000 7f696effd700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f695c0a1480 0x7f695c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.305+0000 7f696effd700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f695c0a1480 0x7f695c0a3940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6960000c00 tx=0x7f6960003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.308+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6968095ef0 con 0x7f6970103040 2023-12-11T19:33:43.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.526+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6970068230 con 0x7f6970103040 2023-12-11T19:33:43.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.527+0000 7f696cff9700 1 -- 172.21.15.66:0/1610761995 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f6968099510 con 0x7f6970103040 2023-12-11T19:33:43.528 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:43.528 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:43.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f695c0a1480 msgr2=0x7f695c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f695c0a1480 0x7f695c0a3940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6960000c00 tx=0x7f6960003680 comp rx=0 tx=0).stop 2023-12-11T19:33:43.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 msgr2=0x7f697019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 0x7f697019e8e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f696800ef30 tx=0x7f696800c3a0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 shutdown_connections 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f695c0a1480 0x7f695c0a3940 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6970103040 0x7f697019e8e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69701014b0 0x7f69701973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 --2- 172.21.15.66:0/1610761995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69701026a0 0x7f6970197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 >> 172.21.15.66:0/1610761995 conn(0x7f69700fca20 msgr2=0x7f6970105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 shutdown_connections 2023-12-11T19:33:43.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.530+0000 7f69755ff700 1 -- 172.21.15.66:0/1610761995 wait complete. 2023-12-11T19:33:43.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:33:43.550 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:33:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 -- 172.21.15.66:0/4125216813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1944102790 msgr2=0x7f1944102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 --2- 172.21.15.66:0/4125216813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1944102790 0x7f1944102bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f19380099c0 tx=0x7f1938009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 -- 172.21.15.66:0/4125216813 shutdown_connections 2023-12-11T19:33:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 --2- 172.21.15.66:0/4125216813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19441015a0 0x7f19441019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 --2- 172.21.15.66:0/4125216813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1944103130 0x7f19441099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 --2- 172.21.15.66:0/4125216813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1944102790 0x7f1944102bf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 -- 172.21.15.66:0/4125216813 >> 172.21.15.66:0/4125216813 conn(0x7f19440fcb10 msgr2=0x7f19440fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 -- 172.21.15.66:0/4125216813 shutdown_connections 2023-12-11T19:33:43.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.763+0000 7f194a5ff700 1 -- 172.21.15.66:0/4125216813 wait complete. 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 Processor -- start 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 -- start start 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 0x7f1944199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1944102790 0x7f1944199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1944103130 0x7f19441a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441a11a0 con 0x7f19441015a0 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441a12e0 con 0x7f1944103130 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.764+0000 7f194a5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441a1470 con 0x7f1944102790 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 0x7f1944199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 0x7f1944199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50840/0 (socket says 172.21.15.66:50840) 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 -- 172.21.15.66:0/1765222709 learned_addr learned my addr 172.21.15.66:0/1765222709 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f19437fe700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1944102790 0x7f1944199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1948b9c700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1944103130 0x7f19441a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 -- 172.21.15.66:0/1765222709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1944102790 msgr2=0x7f1944199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1944102790 0x7f1944199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 -- 172.21.15.66:0/1765222709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1944103130 msgr2=0x7f19441a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1944103130 0x7f19441a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.765+0000 7f1943fff700 1 -- 172.21.15.66:0/1765222709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380095f0 con 0x7f19441015a0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.766+0000 7f1943fff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 0x7f1944199670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f193400d980 tx=0x7f193400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.766+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400f890 con 0x7f19441015a0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.766+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193400fe80 con 0x7f19441015a0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.766+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19441a1750 con 0x7f19441015a0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.766+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934019360 con 0x7f19441015a0 2023-12-11T19:33:43.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.766+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19441a1cb0 con 0x7f19441015a0 2023-12-11T19:33:43.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.767+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1944068310 con 0x7f19441015a0 2023-12-11T19:33:43.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.769+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f193400f9f0 con 0x7f19441015a0 2023-12-11T19:33:43.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.770+0000 7f19417fa700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f192c0a1500 0x7f192c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:43.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.770+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f19340cc6e0 con 0x7f19441015a0 2023-12-11T19:33:43.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.771+0000 7f19437fe700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f192c0a1500 0x7f192c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:43.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.771+0000 7f19437fe700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f192c0a1500 0x7f192c0a39c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1938009990 tx=0x7f1938000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:43.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.774+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1934095d00 con 0x7f19441015a0 2023-12-11T19:33:43.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.989+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f194410cf40 con 0x7f19441015a0 2023-12-11T19:33:43.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.989+0000 7f19417fa700 1 -- 172.21.15.66:0/1765222709 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1934099320 con 0x7f19441015a0 2023-12-11T19:33:43.990 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.991+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f192c0a1500 msgr2=0x7f192c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.991+0000 7f194a5ff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f192c0a1500 0x7f192c0a39c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1938009990 tx=0x7f1938000bc0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.991+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 msgr2=0x7f1944199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 0x7f1944199670 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f193400d980 tx=0x7f193400dc90 comp rx=0 tx=0).stop 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 shutdown_connections 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f192c0a1500 0x7f192c0a39c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1944103130 0x7f19441a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19441015a0 0x7f1944199670 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 --2- 172.21.15.66:0/1765222709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1944102790 0x7f1944199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:43.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 >> 172.21.15.66:0/1765222709 conn(0x7f19440fcb10 msgr2=0x7f1944107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:43.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 shutdown_connections 2023-12-11T19:33:43.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:43.992+0000 7f194a5ff700 1 -- 172.21.15.66:0/1765222709 wait complete. 2023-12-11T19:33:44.011 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:33:44.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:43 smithi070 ceph-mon[104044]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-12-11T19:33:44.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:43 smithi070 ceph-mon[104044]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:43 smithi070 ceph-mon[104044]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:44.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:43 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1610761995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:44.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 -- 172.21.15.66:0/3476232749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e41015a0 msgr2=0x7f00e41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/3476232749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e41015a0 0x7f00e41019c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00d40099c0 tx=0x7f00d4009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 -- 172.21.15.66:0/3476232749 shutdown_connections 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/3476232749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e4103130 0x7f00e41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/3476232749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e41015a0 0x7f00e41019c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/3476232749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00e4102790 0x7f00e4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 -- 172.21.15.66:0/3476232749 >> 172.21.15.66:0/3476232749 conn(0x7f00e40fcb10 msgr2=0x7f00e40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 -- 172.21.15.66:0/3476232749 shutdown_connections 2023-12-11T19:33:44.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.222+0000 7f00ea5ff700 1 -- 172.21.15.66:0/3476232749 wait complete. 2023-12-11T19:33:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.223+0000 7f00ea5ff700 1 Processor -- start 2023-12-11T19:33:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.223+0000 7f00ea5ff700 1 -- start start 2023-12-11T19:33:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00ea5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00e41015a0 0x7f00e4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00ea5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 0x7f00e4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00ea5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e4103130 0x7f00e41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00ea5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e41a1270 con 0x7f00e4102790 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00ea5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e41a13b0 con 0x7f00e4103130 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00ea5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e41a1540 con 0x7f00e41015a0 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 0x7f00e4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 0x7f00e4199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50850/0 (socket says 172.21.15.66:50850) 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 -- 172.21.15.66:0/2145317354 learned_addr learned my addr 172.21.15.66:0/2145317354 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e3fff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00e41015a0 0x7f00e4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 -- 172.21.15.66:0/2145317354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00e41015a0 msgr2=0x7f00e4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00e41015a0 0x7f00e4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 -- 172.21.15.66:0/2145317354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e4103130 msgr2=0x7f00e41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e4103130 0x7f00e41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 -- 172.21.15.66:0/2145317354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d40095f0 con 0x7f00e4102790 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.224+0000 7f00e37fe700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 0x7f00e4199c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d950 tx=0x7f00d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.225+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8009940 con 0x7f00e4102790 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.225+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00d8010460 con 0x7f00e4102790 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.225+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800f620 con 0x7f00e4102790 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.225+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e41a1820 con 0x7f00e4102790 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.225+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e41a1d30 con 0x7f00e4102790 2023-12-11T19:33:44.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.226+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e4068310 con 0x7f00e4102790 2023-12-11T19:33:44.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.228+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f00d80105d0 con 0x7f00e4102790 2023-12-11T19:33:44.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.229+0000 7f00e17fa700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f00cc0a1500 0x7f00cc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.230+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f00d80ccae0 con 0x7f00e4102790 2023-12-11T19:33:44.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.230+0000 7f00e3fff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f00cc0a1500 0x7f00cc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.231+0000 7f00e3fff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f00cc0a1500 0x7f00cc0a39c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00d4006010 tx=0x7f00d4003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:44.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.234+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f00d8096140 con 0x7f00e4102790 2023-12-11T19:33:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:43 smithi167 ceph-mon[102750]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-12-11T19:33:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:43 smithi167 ceph-mon[102750]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:43 smithi167 ceph-mon[102750]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:43 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1610761995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:44.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:43 smithi066 ceph-mon[97920]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-12-11T19:33:44.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:43 smithi066 ceph-mon[97920]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:44.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:43 smithi066 ceph-mon[97920]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T19:33:44.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:43 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1610761995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:44.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.455+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00e41a2090 con 0x7f00e4102790 2023-12-11T19:33:44.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.455+0000 7f00e17fa700 1 -- 172.21.15.66:0/2145317354 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00d8016020 con 0x7f00e4102790 2023-12-11T19:33:44.455 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:33:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.457+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f00cc0a1500 msgr2=0x7f00cc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.457+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f00cc0a1500 0x7f00cc0a39c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00d4006010 tx=0x7f00d4003740 comp rx=0 tx=0).stop 2023-12-11T19:33:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 msgr2=0x7f00e4199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 0x7f00e4199c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d950 tx=0x7f00d800dc60 comp rx=0 tx=0).stop 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 shutdown_connections 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f00cc0a1500 0x7f00cc0a39c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e4103130 0x7f00e41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00e41015a0 0x7f00e4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 --2- 172.21.15.66:0/2145317354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00e4102790 0x7f00e4199c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 >> 172.21.15.66:0/2145317354 conn(0x7f00e40fcb10 msgr2=0x7f00e4107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 shutdown_connections 2023-12-11T19:33:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.458+0000 7f00ea5ff700 1 -- 172.21.15.66:0/2145317354 wait complete. 2023-12-11T19:33:44.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:33:44.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 -- 172.21.15.66:0/2948352092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d8103130 msgr2=0x7f83d81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 --2- 172.21.15.66:0/2948352092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d8103130 0x7f83d81099e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f83d000b0d0 tx=0x7f83d000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 -- 172.21.15.66:0/2948352092 shutdown_connections 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 --2- 172.21.15.66:0/2948352092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d8103130 0x7f83d81099e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 --2- 172.21.15.66:0/2948352092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d81015a0 0x7f83d81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 --2- 172.21.15.66:0/2948352092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83d8102790 0x7f83d8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 -- 172.21.15.66:0/2948352092 >> 172.21.15.66:0/2948352092 conn(0x7f83d80fcb10 msgr2=0x7f83d80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 -- 172.21.15.66:0/2948352092 shutdown_connections 2023-12-11T19:33:44.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.688+0000 7f83de7ff700 1 -- 172.21.15.66:0/2948352092 wait complete. 2023-12-11T19:33:44.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.689+0000 7f83de7ff700 1 Processor -- start 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.689+0000 7f83de7ff700 1 -- start start 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83de7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 0x7f83d8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83de7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d8102790 0x7f83d8199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83de7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83d8103130 0x7f83d81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83de7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d81a1270 con 0x7f83d81015a0 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83de7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d81a13b0 con 0x7f83d8103130 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83de7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d81a1540 con 0x7f83d8102790 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 0x7f83d8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 0x7f83d8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50856/0 (socket says 172.21.15.66:50856) 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 -- 172.21.15.66:0/805773816 learned_addr learned my addr 172.21.15.66:0/805773816 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83dcd9c700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83d8103130 0x7f83d81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d77fe700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d8102790 0x7f83d8199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 -- 172.21.15.66:0/805773816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d8102790 msgr2=0x7f83d8199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d8102790 0x7f83d8199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 -- 172.21.15.66:0/805773816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83d8103130 msgr2=0x7f83d81a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83d8103130 0x7f83d81a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.690+0000 7f83d7fff700 1 -- 172.21.15.66:0/805773816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0009d00 con 0x7f83d81015a0 2023-12-11T19:33:44.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.691+0000 7f83d7fff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 0x7f83d8199740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f83c800d980 tx=0x7f83c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:44.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.691+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800de60 con 0x7f83d81015a0 2023-12-11T19:33:44.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.691+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83c80044b0 con 0x7f83d81015a0 2023-12-11T19:33:44.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.691+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8003b40 con 0x7f83d81015a0 2023-12-11T19:33:44.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.691+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d81a1820 con 0x7f83d81015a0 2023-12-11T19:33:44.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.691+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d81a1d30 con 0x7f83d81015a0 2023-12-11T19:33:44.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.692+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d8068310 con 0x7f83d81015a0 2023-12-11T19:33:44.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.694+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f83c8004870 con 0x7f83d81015a0 2023-12-11T19:33:44.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.696+0000 7f83d57fa700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f83c00a1500 0x7f83c00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:44.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.696+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f83c80ccce0 con 0x7f83d81015a0 2023-12-11T19:33:44.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.696+0000 7f83d77fe700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f83c00a1500 0x7f83c00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:44.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.697+0000 7f83d77fe700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f83c00a1500 0x7f83c00a39c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009bf0 tx=0x7f83cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:44.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.698+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f83c8096380 con 0x7f83d81015a0 2023-12-11T19:33:44.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.916+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83d81a20b0 con 0x7f83d81015a0 2023-12-11T19:33:44.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.917+0000 7f83d57fa700 1 -- 172.21.15.66:0/805773816 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83c80170a0 con 0x7f83d81015a0 2023-12-11T19:33:44.917 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f83c00a1500 msgr2=0x7f83c00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f83c00a1500 0x7f83c00a39c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009bf0 tx=0x7f83cc009260 comp rx=0 tx=0).stop 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 msgr2=0x7f83d8199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 0x7f83d8199740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f83c800d980 tx=0x7f83c800dd40 comp rx=0 tx=0).stop 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 shutdown_connections 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f83c00a1500 0x7f83c00a39c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83d8103130 0x7f83d81a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83d81015a0 0x7f83d8199740 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 --2- 172.21.15.66:0/805773816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d8102790 0x7f83d8199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:44.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.919+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 >> 172.21.15.66:0/805773816 conn(0x7f83d80fcb10 msgr2=0x7f83d8107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:44.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.920+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 shutdown_connections 2023-12-11T19:33:44.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:44.920+0000 7f83de7ff700 1 -- 172.21.15.66:0/805773816 wait complete. 2023-12-11T19:33:44.939 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:33:45.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:44 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1765222709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:45.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:44 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2145317354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 -- 172.21.15.66:0/1890958026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0109360 msgr2=0x7f66c010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 --2- 172.21.15.66:0/1890958026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0109360 0x7f66c010b7d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009990 tx=0x7f66b8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 -- 172.21.15.66:0/1890958026 shutdown_connections 2023-12-11T19:33:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 --2- 172.21.15.66:0/1890958026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c00755b0 0x7f66c0073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 --2- 172.21.15.66:0/1890958026 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0109360 0x7f66c010b7d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 --2- 172.21.15.66:0/1890958026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0073fd0 0x7f66c0074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 -- 172.21.15.66:0/1890958026 >> 172.21.15.66:0/1890958026 conn(0x7f66c00fca00 msgr2=0x7f66c00fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 -- 172.21.15.66:0/1890958026 shutdown_connections 2023-12-11T19:33:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.146+0000 7f66c7dff700 1 -- 172.21.15.66:0/1890958026 wait complete. 2023-12-11T19:33:45.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 Processor -- start 2023-12-11T19:33:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 -- start start 2023-12-11T19:33:45.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 0x7f66c01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c00755b0 0x7f66c01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0109360 0x7f66c01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01a0fe0 con 0x7f66c0073fd0 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01a1120 con 0x7f66c00755b0 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.148+0000 7f66c7dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01a12b0 con 0x7f66c0109360 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 0x7f66c01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 0x7f66c01994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50862/0 (socket says 172.21.15.66:50862) 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 -- 172.21.15.66:0/4159591639 learned_addr learned my addr 172.21.15.66:0/4159591639 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c539a700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c00755b0 0x7f66c01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c639c700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0109360 0x7f66c01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 -- 172.21.15.66:0/4159591639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0109360 msgr2=0x7f66c01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0109360 0x7f66c01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 -- 172.21.15.66:0/4159591639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c00755b0 msgr2=0x7f66c01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c00755b0 0x7f66c01999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 -- 172.21.15.66:0/4159591639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80095f0 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c639c700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0109360 0x7f66c01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.149+0000 7f66c5b9b700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 0x7f66c01994b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ef10 tx=0x7f66bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.150+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00ca70 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.150+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66bc00f040 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.150+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc014480 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.150+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c01a1530 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.150+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c01a19f0 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.151+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c0108170 con 0x7f66c0073fd0 2023-12-11T19:33:45.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.153+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f66bc007500 con 0x7f66c0073fd0 2023-12-11T19:33:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.154+0000 7f66b6ffd700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66ac0a1500 0x7f66ac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.154+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66bc0ccfe0 con 0x7f66c0073fd0 2023-12-11T19:33:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.154+0000 7f66c539a700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66ac0a1500 0x7f66ac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.155+0000 7f66c539a700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66ac0a1500 0x7f66ac0a39c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009c60 tx=0x7f66b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:45.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.158+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f66bc096640 con 0x7f66c0073fd0 2023-12-11T19:33:45.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:44 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1765222709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:45.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:44 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2145317354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:45.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:44 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1765222709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:33:45.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:44 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2145317354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:33:45.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.418+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66c0068680 con 0x7f66c0073fd0 2023-12-11T19:33:45.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.419+0000 7f66b6ffd700 1 -- 172.21.15.66:0/4159591639 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f66bc099c60 con 0x7f66c0073fd0 2023-12-11T19:33:45.419 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:45.419 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:33:45.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66ac0a1500 msgr2=0x7f66ac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66ac0a1500 0x7f66ac0a39c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009c60 tx=0x7f66b0009260 comp rx=0 tx=0).stop 2023-12-11T19:33:45.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 msgr2=0x7f66c01994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 0x7f66c01994b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ef10 tx=0x7f66bc00c3a0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 shutdown_connections 2023-12-11T19:33:45.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66ac0a1500 0x7f66ac0a39c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66c0109360 0x7f66c01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66c0073fd0 0x7f66c01994b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 --2- 172.21.15.66:0/4159591639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c00755b0 0x7f66c01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 >> 172.21.15.66:0/4159591639 conn(0x7f66c00fca00 msgr2=0x7f66c01052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 shutdown_connections 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.422+0000 7f66c7dff700 1 -- 172.21.15.66:0/4159591639 wait complete. 2023-12-11T19:33:45.423 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:33:45.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:33:45.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 -- 172.21.15.66:0/3021797376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 msgr2=0x7f52bc1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 --2- 172.21.15.66:0/3021797376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc1095d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b0a0 tx=0x7f52b400b3b0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 -- 172.21.15.66:0/3021797376 shutdown_connections 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 --2- 172.21.15.66:0/3021797376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc1095d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 --2- 172.21.15.66:0/3021797376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc101f00 0x7f52bc1042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 --2- 172.21.15.66:0/3021797376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52bc104830 0x7f52bc106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 -- 172.21.15.66:0/3021797376 >> 172.21.15.66:0/3021797376 conn(0x7f52bc0fb8f0 msgr2=0x7f52bc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 -- 172.21.15.66:0/3021797376 shutdown_connections 2023-12-11T19:33:45.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.649+0000 7f52c35ff700 1 -- 172.21.15.66:0/3021797376 wait complete. 2023-12-11T19:33:45.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.650+0000 7f52c35ff700 1 Processor -- start 2023-12-11T19:33:45.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.650+0000 7f52c35ff700 1 -- start start 2023-12-11T19:33:45.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c35ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52bc101f00 0x7f52bc1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c35ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc104830 0x7f52bc1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c35ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c35ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19eee0 con 0x7f52bc107160 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c35ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19f020 con 0x7f52bc101f00 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c35ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19f1b0 con 0x7f52bc104830 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50876/0 (socket says 172.21.15.66:50876) 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 -- 172.21.15.66:0/1789545585 learned_addr learned my addr 172.21.15.66:0/1789545585 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c0b9a700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc104830 0x7f52bc1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c139b700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52bc101f00 0x7f52bc1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 -- 172.21.15.66:0/1789545585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc104830 msgr2=0x7f52bc1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc104830 0x7f52bc1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 -- 172.21.15.66:0/1789545585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52bc101f00 msgr2=0x7f52bc1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52bc101f00 0x7f52bc1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c1b9c700 1 -- 172.21.15.66:0/1789545585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009d00 con 0x7f52bc107160 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.651+0000 7f52c0b9a700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc104830 0x7f52bc1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.652+0000 7f52c1b9c700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc19e910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b780 tx=0x7f52b400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:45.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.652+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400e040 con 0x7f52bc107160 2023-12-11T19:33:45.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.652+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52b400be00 con 0x7f52bc107160 2023-12-11T19:33:45.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.652+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40052a0 con 0x7f52bc107160 2023-12-11T19:33:45.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.652+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc19f430 con 0x7f52bc107160 2023-12-11T19:33:45.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.652+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc19f800 con 0x7f52bc107160 2023-12-11T19:33:45.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.653+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc0fd470 con 0x7f52bc107160 2023-12-11T19:33:45.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.655+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f52b401d9a0 con 0x7f52bc107160 2023-12-11T19:33:45.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.657+0000 7f52b27fc700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52a80a1500 0x7f52a80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:45.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.657+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f52b40cdf10 con 0x7f52bc107160 2023-12-11T19:33:45.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.657+0000 7f52c139b700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52a80a1500 0x7f52a80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:45.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.658+0000 7f52c139b700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52a80a1500 0x7f52a80a39c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009bf0 tx=0x7f52b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:45.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.661+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f52b40975c0 con 0x7f52bc107160 2023-12-11T19:33:45.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.878+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52bc068230 con 0x7f52bc107160 2023-12-11T19:33:45.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.880+0000 7f52b27fc700 1 -- 172.21.15.66:0/1789545585 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f52b409abe0 con 0x7f52bc107160 2023-12-11T19:33:45.880 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:33:45.881 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52a80a1500 msgr2=0x7f52a80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52a80a1500 0x7f52a80a39c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009bf0 tx=0x7f52b8009260 comp rx=0 tx=0).stop 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 msgr2=0x7f52bc19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc19e910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b780 tx=0x7f52b400bb60 comp rx=0 tx=0).stop 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 shutdown_connections 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f52a80a1500 0x7f52a80a39c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52bc101f00 0x7f52bc1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52bc107160 0x7f52bc19e910 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 --2- 172.21.15.66:0/1789545585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52bc104830 0x7f52bc1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.882+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 >> 172.21.15.66:0/1789545585 conn(0x7f52bc0fb8f0 msgr2=0x7f52bc102860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:45.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.883+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 shutdown_connections 2023-12-11T19:33:45.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:45.883+0000 7f52c35ff700 1 -- 172.21.15.66:0/1789545585 wait complete. 2023-12-11T19:33:45.903 DEBUG:tasks.fwd_scrub:Yielding 2023-12-11T19:33:45.903 INFO:teuthology.run_tasks:Running task background_exec... 2023-12-11T19:33:45.925 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:33:45.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-12-11T19:33:45.926 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-12-11T19:33:45.926 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-11T19:33:45.927 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-11T19:33:45.932 INFO:teuthology.run_tasks:Running task workunit... 2023-12-11T19:33:45.979 INFO:tasks.workunit:Pulling workunits from ref a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:33:45.979 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-11T19:33:45.979 INFO:tasks.workunit:timeout=3h 2023-12-11T19:33:45.979 INFO:tasks.workunit:cleanup=True 2023-12-11T19:33:45.980 DEBUG:teuthology.orchestra.run.smithi066:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 79h/121d Inode: 1 Links: 2 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:fusefs_t:s0 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-12-11 19:31:58.034957073 +0000 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-12-11 19:31:58.034957073 +0000 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-12-11 19:33:37.355451343 +0000 2023-12-11T19:33:46.033 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2023-12-11T19:33:46.033 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-11T19:33:46.034 DEBUG:teuthology.orchestra.run.smithi066:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-11T19:33:46.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:45 smithi070 ceph-mon[104044]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-12-11T19:33:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:45 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/805773816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:46.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:45 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4159591639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:46.137 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:45 smithi066 ceph-mon[97920]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-12-11T19:33:46.138 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:45 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/805773816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:46.138 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:45 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4159591639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.148+0000 7f7f23fff700 1 -- 172.21.15.66:0/4084001869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 msgr2=0x7f7f1c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.148+0000 7f7f23fff700 1 --2- 172.21.15.66:0/4084001869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c0744b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009990 tx=0x7f7f0c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 -- 172.21.15.66:0/4084001869 shutdown_connections 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 --2- 172.21.15.66:0/4084001869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 0x7f7f1c073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 --2- 172.21.15.66:0/4084001869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f1c10db70 0x7f7f1c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 --2- 172.21.15.66:0/4084001869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c0744b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 -- 172.21.15.66:0/4084001869 >> 172.21.15.66:0/4084001869 conn(0x7f7f1c0fcb10 msgr2=0x7f7f1c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 -- 172.21.15.66:0/4084001869 shutdown_connections 2023-12-11T19:33:46.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.149+0000 7f7f23fff700 1 -- 172.21.15.66:0/4084001869 wait complete. 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 Processor -- start 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 -- start start 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 0x7f7f1c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f1c10db70 0x7f7f1c1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c1a5570 con 0x7f7f1c074030 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c1a56b0 con 0x7f7f1c075610 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f23fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c1a5840 con 0x7f7f1c10db70 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f21d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f21d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50902/0 (socket says 172.21.15.66:50902) 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.150+0000 7f7f21d9b700 1 -- 172.21.15.66:0/1630378908 learned_addr learned my addr 172.21.15.66:0/1630378908 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f2159a700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 0x7f7f1c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f1c10db70 0x7f7f1c1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f21d9b700 1 -- 172.21.15.66:0/1630378908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f1c10db70 msgr2=0x7f7f1c1a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f1c10db70 0x7f7f1c1a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f21d9b700 1 -- 172.21.15.66:0/1630378908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 msgr2=0x7f7f1c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 0x7f7f1c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f21d9b700 1 -- 172.21.15.66:0/1630378908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0095f0 con 0x7f7f1c074030 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f2159a700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 0x7f7f1c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c19da40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800d950 tx=0x7f7f1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f12ffd700 1 -- 172.21.15.66:0/1630378908 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18009940 con 0x7f7f1c074030 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f12ffd700 1 -- 172.21.15.66:0/1630378908 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f18010460 con 0x7f7f1c074030 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.151+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c1a5b20 con 0x7f7f1c074030 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.152+0000 7f7f12ffd700 1 -- 172.21.15.66:0/1630378908 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800f620 con 0x7f7f1c074030 2023-12-11T19:33:46.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.152+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c1a6080 con 0x7f7f1c074030 2023-12-11T19:33:46.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.155+0000 7f7f12ffd700 1 -- 172.21.15.66:0/1630378908 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7f180105d0 con 0x7f7f1c074030 2023-12-11T19:33:46.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.156+0000 7f7f3797db80 1 Processor -- start 2023-12-11T19:33:46.160 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.156+0000 7f7f3797db80 1 -- start start 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f12ffd700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7f080a1500 0x7f7f080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb7362870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 0x560eb73636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7363ca0 0x560eb7364120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f3797db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb72bdbd0 con 0x560eb7362db0 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f3797db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb72bb4f0 con 0x560eb7363ca0 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f3797db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb72b6de0 con 0x560eb7362450 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f12ffd700 1 -- 172.21.15.66:0/1630378908 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f180cce60 con 0x7f7f1c074030 2023-12-11T19:33:46.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f2159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 0x560eb73636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7f080a1500 0x7f7f080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f2159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 0x560eb73636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50914/0 (socket says 172.21.15.66:50914) 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f2159a700 1 -- 172.21.15.66:0/1636875539 learned_addr learned my addr 172.21.15.66:0/1636875539 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.157+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7363ca0 0x560eb7364120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb7362870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2159a700 1 -- 172.21.15.66:0/1636875539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 msgr2=0x560eb7362870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2159a700 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb7362870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2159a700 1 -- 172.21.15.66:0/1636875539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7363ca0 msgr2=0x560eb7364120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2159a700 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7363ca0 0x560eb7364120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2159a700 1 -- 172.21.15.66:0/1636875539 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560eb71336b0 con 0x560eb7362db0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2159a700 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 0x560eb73636c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c075330 tx=0x7f7f0c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37de46a0e49ae11 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7f080a1500 0x7f7f080a39c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x560eb7363480 tx=0x7f7f14003aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f07fff700 1 -- 172.21.15.66:0/1636875539 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00b9d0 con 0x560eb7362db0 2023-12-11T19:33:46.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f07fff700 1 -- 172.21.15.66:0/1636875539 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f0c01e040 con 0x560eb7362db0 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.158+0000 7f7f07fff700 1 -- 172.21.15.66:0/1636875539 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0047f0 con 0x560eb7362db0 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 -- 172.21.15.66:0/1636875539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 msgr2=0x560eb73636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 0x560eb73636c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c075330 tx=0x7f7f0c005e80 comp rx=0 tx=0).stop 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 -- 172.21.15.66:0/1636875539 shutdown_connections 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7363ca0 0x560eb7364120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb7362870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1636875539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7362db0 0x560eb73636c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 -- 172.21.15.66:0/1636875539 >> 172.21.15.66:0/1636875539 conn(0x560eb7291860 msgr2=0x560eb735ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 -- 172.21.15.66:0/1636875539 shutdown_connections 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.159+0000 7f7f3797db80 1 -- 172.21.15.66:0/1636875539 wait complete. 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.160+0000 7f7f3797db80 1 -- start start 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.160+0000 7f7f3797db80 1 Processor -- start 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb736c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb736c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb7373750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb72705e0 con 0x560eb7363ca0 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb7270280 con 0x560eb7362450 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb71359a0 con 0x560eb7362db0 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f3797db80 10 client.0 _ll_register_callbacks cb 0x560eb719bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb7373750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb7373750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50930/0 (socket says 172.21.15.66:50930) 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 -- 172.21.15.66:0/3758153129 learned_addr learned my addr 172.21.15.66:0/3758153129 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2159a700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb736c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb736c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 -- 172.21.15.66:0/3758153129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 msgr2=0x560eb736c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb736c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 -- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 msgr2=0x560eb736c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb736c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.161+0000 7f7f2259c700 1 -- 172.21.15.66:0/3758153129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c110910 con 0x560eb7363ca0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f2259c700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb7373750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7f14019840 tx=0x7f7f14009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f87a54226eba94 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f057fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f057fa700 1 -- 172.21.15.66:0/3758153129 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14008980 con 0x560eb7363ca0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560eb7355bc0 con 0x560eb7363ca0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f057fa700 1 -- 172.21.15.66:0/3758153129 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f14008ae0 con 0x560eb7363ca0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f057fa700 1 -- 172.21.15.66:0/3758153129 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f14008ca0 con 0x560eb7363ca0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f3797db80 10 client.15501 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.162+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560eb70dfe60 con 0x560eb7363ca0 2023-12-11T19:33:46.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f057fa700 1 -- 172.21.15.66:0/3758153129 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560eb7355bc0 con 0x560eb7363ca0 2023-12-11T19:33:46.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f057fa700 1 -- 172.21.15.66:0/3758153129 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7f140233a0 con 0x560eb7363ca0 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f3797db80 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f3797db80 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f3797db80 20 client.15501 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73780a0 0x560eb737a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f3797db80 4 client.15501 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.163+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560eb7331fc0 con 0x560eb73780a0 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.164+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73780a0 0x560eb737a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.164+0000 7f7f057fa700 10 client.15501 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.164+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73780a0 0x560eb737a4e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e583bb3b40e70d9d server_cookie=1d89090da93f27ff in_seq=0 out_seq=0 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f057fa700 1 -- 172.21.15.66:0/3758153129 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560eb7331fc0 con 0x560eb73780a0 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f057fa700 10 client.15501 handle_command_reply: tid=0 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 client.15501 shutdown 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 msgr2=0x560eb7373750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb7373750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7f14019840 tx=0x7f7f14009100 comp rx=0 tx=0).stop 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 shutdown_connections 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73780a0 0x560eb737a4e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb736c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f057fa700 0 client.15501 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb7373750 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3758153129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb736c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.165+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 >> 172.21.15.66:0/3758153129 conn(0x560eb7291860 msgr2=0x560eb735dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.166+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 shutdown_connections 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.166+0000 7f7f3797db80 1 -- 172.21.15.66:0/3758153129 wait complete. 2023-12-11T19:33:46.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.166+0000 7f7f3797db80 20 client.15501 trim_cache size 0 max 16384 2023-12-11T19:33:46.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.186+0000 7f7f3797db80 1 Processor -- start 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.186+0000 7f7f3797db80 1 -- start start 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.186+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb735f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.186+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362db0 0x560eb7373670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.186+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb736bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f3797db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb73748d0 con 0x560eb7363ca0 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f3797db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb736bd40 con 0x560eb7362db0 2023-12-11T19:33:46.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f3797db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb7373bb0 con 0x560eb7362450 2023-12-11T19:33:46.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb736bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb736bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50942/0 (socket says 172.21.15.66:50942) 2023-12-11T19:33:46.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 -- 172.21.15.66:0/3451178056 learned_addr learned my addr 172.21.15.66:0/3451178056 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2159a700 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362db0 0x560eb7373670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb735f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 -- 172.21.15.66:0/3451178056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 msgr2=0x560eb735f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb735f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 -- 172.21.15.66:0/3451178056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362db0 msgr2=0x560eb7373670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362db0 0x560eb7373670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.187+0000 7f7f2259c700 1 -- 172.21.15.66:0/3451178056 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560eb70c8ce0 con 0x560eb7363ca0 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.188+0000 7f7f2259c700 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb736bf80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000f80 tx=0x7f7f14003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e1e4865f3df110 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.188+0000 7f7f10ff9700 1 -- 172.21.15.66:0/3451178056 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14009c60 con 0x560eb7363ca0 2023-12-11T19:33:46.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.188+0000 7f7f10ff9700 1 -- 172.21.15.66:0/3451178056 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f1400f040 con 0x560eb7363ca0 2023-12-11T19:33:46.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.188+0000 7f7f10ff9700 1 -- 172.21.15.66:0/3451178056 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14008b80 con 0x560eb7363ca0 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 -- 172.21.15.66:0/3451178056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 msgr2=0x560eb736bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb736bf80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000f80 tx=0x7f7f14003a00 comp rx=0 tx=0).stop 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 -- 172.21.15.66:0/3451178056 shutdown_connections 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb736bf80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362450 0x560eb735f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 --2- 172.21.15.66:0/3451178056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362db0 0x560eb7373670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 -- 172.21.15.66:0/3451178056 >> 172.21.15.66:0/3451178056 conn(0x560eb7291860 msgr2=0x560eb735e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 -- 172.21.15.66:0/3451178056 shutdown_connections 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 -- 172.21.15.66:0/3451178056 wait complete. 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.189+0000 7f7f3797db80 1 -- start start 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 Processor -- start 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb7371e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb7372390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb73728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb73ce5f0 con 0x560eb7363ca0 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb73ce730 con 0x560eb7362450 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560eb73ce870 con 0x560eb7362db0 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f3797db80 10 client.0 _ll_register_callbacks cb 0x560eb719bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb73728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.190+0000 7f7f2159a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb7372390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb73728d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50944/0 (socket says 172.21.15.66:50944) 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 -- 172.21.15.66:0/1480673786 learned_addr learned my addr 172.21.15.66:0/1480673786 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb7371e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 -- 172.21.15.66:0/1480673786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 msgr2=0x560eb7372390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb7372390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 -- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 msgr2=0x560eb7371e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb7371e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 -- 172.21.15.66:0/1480673786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560eb70dfe60 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2159a700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb7372390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb7371e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f2259c700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb73728d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7f14003a70 tx=0x7f7f140235d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919dec3667888c2a server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.191+0000 7f7f057fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f057fa700 1 -- 172.21.15.66:0/1480673786 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14023d30 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f0c01e040 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f057fa700 1 -- 172.21.15.66:0/1480673786 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f14004bb0 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f057fa700 1 -- 172.21.15.66:0/1480673786 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f14004db0 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f3797db80 10 client.15507 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560eb71062d0 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f057fa700 1 -- 172.21.15.66:0/1480673786 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f0c01e040 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f057fa700 1 -- 172.21.15.66:0/1480673786 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7f14036030 con 0x560eb7363ca0 2023-12-11T19:33:46.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f3797db80 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f3797db80 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:33:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.192+0000 7f7f3797db80 20 client.15507 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.193+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73cfea0 0x560eb73d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.193+0000 7f7f3797db80 4 client.15507 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-11T19:33:46.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.193+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560eb727d050 con 0x560eb73cfea0 2023-12-11T19:33:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.193+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73cfea0 0x560eb73d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.194+0000 7f7f21d9b700 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73cfea0 0x560eb73d22c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02352cb5d9787b9 server_cookie=5aed13df99b4c29e in_seq=0 out_seq=0 2023-12-11T19:33:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.194+0000 7f7f057fa700 10 client.15507 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f057fa700 1 -- 172.21.15.66:0/1480673786 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560eb727d050 con 0x560eb73cfea0 2023-12-11T19:33:46.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f057fa700 10 client.15507 handle_command_reply: tid=0 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 client.15507 shutdown 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 msgr2=0x560eb73728d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb73728d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7f14003a70 tx=0x7f7f140235d0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 shutdown_connections 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560eb73cfea0 0x560eb73d22c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560eb7362450 0x560eb7371e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560eb7363ca0 0x560eb73728d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 --2- 172.21.15.66:0/1480673786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560eb7362db0 0x560eb7372390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 >> 172.21.15.66:0/1480673786 conn(0x560eb7291860 msgr2=0x560eb736bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.195+0000 7f7f057fa700 0 client.15507 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 shutdown_connections 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f3797db80 1 -- 172.21.15.66:0/1480673786 wait complete. 2023-12-11T19:33:46.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f3797db80 20 client.15507 trim_cache size 0 max 16384 2023-12-11T19:33:46.199 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:46.199 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-12-11T19:33:46.199 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "69c12c31-b341-488c-94ae-23365c83e7ba", 2023-12-11T19:33:46.199 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-12-11T19:33:46.199 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7f080a1500 msgr2=0x7f7f080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f23fff700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7f080a1500 0x7f7f080a39c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x560eb7363480 tx=0x7f7f14003aa0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 msgr2=0x7f7f1c19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.196+0000 7f7f23fff700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c19da40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800d950 tx=0x7f7f1800dc60 comp rx=0 tx=0).stop 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 shutdown_connections 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7f080a1500 0x7f7f080a39c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f1c10db70 0x7f7f1c1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f1c074030 0x7f7f1c19da40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 --2- 172.21.15.66:0/1630378908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c075610 0x7f7f1c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 >> 172.21.15.66:0/1630378908 conn(0x7f7f1c0fcb10 msgr2=0x7f7f1c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 shutdown_connections 2023-12-11T19:33:46.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.197+0000 7f7f23fff700 1 -- 172.21.15.66:0/1630378908 wait complete. 2023-12-11T19:33:46.212 INFO:tasks.workunit.client.0.smithi066.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-11T19:33:46.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:33:46.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:45 smithi167 ceph-mon[102750]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-12-11T19:33:46.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:45 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/805773816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:33:46.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:45 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4159591639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:33:46.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.469+0000 7f49a7dff700 1 -- 172.21.15.66:0/1214737997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0101680 msgr2=0x7f49a0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.469+0000 7f49a7dff700 1 --2- 172.21.15.66:0/1214737997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0101680 0x7f49a0101ae0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f49900099c0 tx=0x7f4990009cd0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 -- 172.21.15.66:0/1214737997 shutdown_connections 2023-12-11T19:33:46.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 --2- 172.21.15.66:0/1214737997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0102020 0x7f49a01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 --2- 172.21.15.66:0/1214737997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a0100490 0x7f49a01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 --2- 172.21.15.66:0/1214737997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0101680 0x7f49a0101ae0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 -- 172.21.15.66:0/1214737997 >> 172.21.15.66:0/1214737997 conn(0x7f49a00fba20 msgr2=0x7f49a00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 -- 172.21.15.66:0/1214737997 shutdown_connections 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.470+0000 7f49a7dff700 1 -- 172.21.15.66:0/1214737997 wait complete. 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.471+0000 7f49a7dff700 1 Processor -- start 2023-12-11T19:33:46.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.471+0000 7f49a7dff700 1 -- start start 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a7dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 0x7f49a01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a7dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0101680 0x7f49a0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a7dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a0102020 0x7f49a01a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a7dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a01a11d0 con 0x7f49a0100490 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a7dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a01a1310 con 0x7f49a0101680 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a7dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a01a14a0 con 0x7f49a0102020 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 0x7f49a01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 0x7f49a01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50960/0 (socket says 172.21.15.66:50960) 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a5b9b700 1 -- 172.21.15.66:0/2968983299 learned_addr learned my addr 172.21.15.66:0/2968983299 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.472+0000 7f49a539a700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0101680 0x7f49a0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a639c700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a0102020 0x7f49a01a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a539a700 1 -- 172.21.15.66:0/2968983299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a0102020 msgr2=0x7f49a01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a539a700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a0102020 0x7f49a01a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a539a700 1 -- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 msgr2=0x7f49a01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a539a700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 0x7f49a01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a539a700 1 -- 172.21.15.66:0/2968983299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c009520 con 0x7f49a0101680 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 0x7f49a01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:46.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a539a700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0101680 0x7f49a0199be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4990009fd0 tx=0x7f499000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f4996ffd700 1 -- 172.21.15.66:0/2968983299 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499001d070 con 0x7f49a0101680 2023-12-11T19:33:46.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.473+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900095f0 con 0x7f49a0101680 2023-12-11T19:33:46.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.474+0000 7f4996ffd700 1 -- 172.21.15.66:0/2968983299 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49900054e0 con 0x7f49a0101680 2023-12-11T19:33:46.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.474+0000 7f4996ffd700 1 -- 172.21.15.66:0/2968983299 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990004b30 con 0x7f49a0101680 2023-12-11T19:33:46.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.474+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a01a1a10 con 0x7f49a0101680 2023-12-11T19:33:46.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.477+0000 7f4996ffd700 1 -- 172.21.15.66:0/2968983299 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4990004c90 con 0x7f49a0101680 2023-12-11T19:33:46.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 Processor -- start 2023-12-11T19:33:46.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 -- start start 2023-12-11T19:33:46.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e7d30 0x559bf15e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e8c20 0x559bf15e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf1542b50 con 0x559bf15e73d0 2023-12-11T19:33:46.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf1540470 con 0x559bf15e8c20 2023-12-11T19:33:46.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.478+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf153bd60 con 0x559bf15e7d30 2023-12-11T19:33:46.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15e77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50974/0 (socket says 172.21.15.66:50974) 2023-12-11T19:33:46.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 -- 172.21.15.66:0/478341224 learned_addr learned my addr 172.21.15.66:0/478341224 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a539a700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e8c20 0x559bf15e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e7d30 0x559bf15e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 -- 172.21.15.66:0/478341224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e7d30 msgr2=0x559bf15e8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e7d30 0x559bf15e8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 -- 172.21.15.66:0/478341224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e8c20 msgr2=0x559bf15e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e8c20 0x559bf15e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a639c700 1 -- 172.21.15.66:0/478341224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bf13b8630 con 0x559bf15e73d0 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a539a700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e8c20 0x559bf15e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.479+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e7d30 0x559bf15e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f4996ffd700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f498c0a1480 0x7f498c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f49a639c700 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15e77f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4998009440 tx=0x7f4998009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b7bb3321e7deb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f4996ffd700 1 -- 172.21.15.66:0/2968983299 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f49900cda00 con 0x7f49a0101680 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f498c0a1480 0x7f498c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f498bfff700 1 -- 172.21.15.66:0/478341224 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800e070 con 0x559bf15e73d0 2023-12-11T19:33:46.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f498bfff700 1 -- 172.21.15.66:0/478341224 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f499800f040 con 0x559bf15e73d0 2023-12-11T19:33:46.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.480+0000 7f498bfff700 1 -- 172.21.15.66:0/478341224 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998015a40 con 0x559bf15e73d0 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/478341224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 msgr2=0x559bf15e77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f498c0a1480 0x7f498c0a3940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f499c009fd0 tx=0x7f499c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15e77f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4998009440 tx=0x7f4998009920 comp rx=0 tx=0).stop 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/478341224 shutdown_connections 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e8c20 0x559bf15e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15e77f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/478341224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e7d30 0x559bf15e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/478341224 >> 172.21.15.66:0/478341224 conn(0x559bf15167e0 msgr2=0x559bf15e3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/478341224 shutdown_connections 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.481+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/478341224 wait complete. 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 -- start start 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 Processor -- start 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 0x559bf15f8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf14f5560 con 0x559bf15e8c20 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf14f5200 con 0x559bf15e7d30 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.482+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf13ba920 con 0x559bf15e73d0 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49bb6d6b80 10 client.0 _ll_register_callbacks cb 0x559bf1420a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 0x559bf15f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:57120/0 (socket says 172.21.15.66:57120) 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a5b9b700 1 -- 172.21.15.66:0/4083783647 learned_addr learned my addr 172.21.15.66:0/4083783647 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 0x559bf15f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50986/0 (socket says 172.21.15.66:50986) 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a539a700 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 -- 172.21.15.66:0/4083783647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e73d0 msgr2=0x559bf15f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 -- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 msgr2=0x559bf15f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.483+0000 7f49a639c700 1 -- 172.21.15.66:0/4083783647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49980090f0 con 0x559bf15e8c20 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49a639c700 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 0x559bf15f8720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4998009440 tx=0x7f4998016a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a51b6f08c3ca1ba server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49897fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49897fa700 1 -- 172.21.15.66:0/4083783647 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800e040 con 0x559bf15e8c20 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bf15dab40 con 0x559bf15e8c20 2023-12-11T19:33:46.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49897fa700 1 -- 172.21.15.66:0/4083783647 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f49980095e0 con 0x559bf15e8c20 2023-12-11T19:33:46.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49897fa700 1 -- 172.21.15.66:0/4083783647 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4998016eb0 con 0x559bf15e8c20 2023-12-11T19:33:46.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49bb6d6b80 10 client.15519 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:46.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.484+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559bf1364de0 con 0x559bf15e8c20 2023-12-11T19:33:46.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49897fa700 1 -- 172.21.15.66:0/4083783647 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559bf15dab40 con 0x559bf15e8c20 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49897fa700 1 -- 172.21.15.66:0/4083783647 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f499801c050 con 0x559bf15e8c20 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49bb6d6b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49bb6d6b80 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49bb6d6b80 20 client.15519 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf15fce10 0x559bf15ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49bb6d6b80 4 client.15519 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bf15b6f40 con 0x559bf15fce10 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.485+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf15fce10 0x559bf15ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.486+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf15fce10 0x559bf15ff230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ebcc0da7979612 server_cookie=9793fb328a53db1f in_seq=0 out_seq=0 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.486+0000 7f49897fa700 10 client.15519 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49897fa700 1 -- 172.21.15.66:0/4083783647 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bf15b6f40 con 0x559bf15fce10 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49897fa700 10 client.15519 handle_command_reply: tid=0 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 client.15519 shutdown 2023-12-11T19:33:46.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 msgr2=0x559bf15f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 0x559bf15f8720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4998009440 tx=0x7f4998016a40 comp rx=0 tx=0).stop 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 shutdown_connections 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf15fce10 0x559bf15ff230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e8c20 0x559bf15f8720 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/4083783647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49897fa700 0 client.15519 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 >> 172.21.15.66:0/4083783647 conn(0x559bf15167e0 msgr2=0x559bf15e2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 shutdown_connections 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.487+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/4083783647 wait complete. 2023-12-11T19:33:46.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.488+0000 7f49bb6d6b80 20 client.15519 trim_cache size 0 max 16384 2023-12-11T19:33:46.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 Processor -- start 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 -- start start 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf15e8110 con 0x559bf15e73d0 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf1604400 con 0x559bf15e7d30 2023-12-11T19:33:46.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.506+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf15ff890 con 0x559bf15e8c20 2023-12-11T19:33:46.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a539a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51000/0 (socket says 172.21.15.66:51000) 2023-12-11T19:33:46.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a5b9b700 1 -- 172.21.15.66:0/435684811 learned_addr learned my addr 172.21.15.66:0/435684811 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a639c700 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a539a700 1 -- 172.21.15.66:0/435684811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 msgr2=0x559bf15f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a539a700 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a539a700 1 -- 172.21.15.66:0/435684811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 msgr2=0x559bf15f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a539a700 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a539a700 1 -- 172.21.15.66:0/435684811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bf134dc60 con 0x559bf15e7d30 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.507+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.508+0000 7f49a539a700 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f86e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x559bf15e8400 tx=0x7f499001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd1aadf9d0d7e0c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.508+0000 7f4994bff700 1 -- 172.21.15.66:0/435684811 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499009a6d0 con 0x559bf15e7d30 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.508+0000 7f4994bff700 1 -- 172.21.15.66:0/435684811 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f499009b470 con 0x559bf15e7d30 2023-12-11T19:33:46.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.508+0000 7f4994bff700 1 -- 172.21.15.66:0/435684811 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900cb5d0 con 0x559bf15e7d30 2023-12-11T19:33:46.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/435684811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 msgr2=0x559bf15f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f86e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x559bf15e8400 tx=0x7f499001f000 comp rx=0 tx=0).stop 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/435684811 shutdown_connections 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e73d0 0x559bf15f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/435684811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e7d30 0x559bf15f86e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/435684811 >> 172.21.15.66:0/435684811 conn(0x559bf15167e0 msgr2=0x559bf15ef170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/435684811 shutdown_connections 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/435684811 wait complete. 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 -- start start 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.509+0000 7f49bb6d6b80 1 Processor -- start 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e73d0 0x559bf15f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 0x559bf15f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf15f81e0 con 0x559bf15e7d30 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf15f8350 con 0x559bf15e73d0 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf15f8510 con 0x559bf15e8c20 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49bb6d6b80 10 client.0 _ll_register_callbacks cb 0x559bf1420a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49a539a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 0x559bf15f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49a5b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e73d0 0x559bf15f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49a539a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 0x559bf15f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51002/0 (socket says 172.21.15.66:51002) 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.510+0000 7f49a539a700 1 -- 172.21.15.66:0/505800765 learned_addr learned my addr 172.21.15.66:0/505800765 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:46.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a639c700 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a539a700 1 -- 172.21.15.66:0/505800765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 msgr2=0x559bf15f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a539a700 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a539a700 1 -- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e73d0 msgr2=0x559bf15f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a539a700 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e73d0 0x559bf15f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a539a700 1 -- 172.21.15.66:0/505800765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c012dc0 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49a539a700 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 0x559bf15f4c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f499000b580 tx=0x7f4990005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b919cedfb65537cf server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.511+0000 7f49897fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49897fa700 1 -- 172.21.15.66:0/505800765 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499009a6d0 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49897fa700 1 -- 172.21.15.66:0/505800765 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f499009ad70 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f499800f040 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49897fa700 1 -- 172.21.15.66:0/505800765 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4990005650 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49bb6d6b80 10 client.15525 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f49a01a1a10 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49897fa700 1 -- 172.21.15.66:0/505800765 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4990064020 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.512+0000 7f49897fa700 1 -- 172.21.15.66:0/505800765 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f49900a2c70 con 0x559bf15e7d30 2023-12-11T19:33:46.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49bb6d6b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49bb6d6b80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:33:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49bb6d6b80 20 client.15525 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf1654110 0x559bf1656530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49bb6d6b80 4 client.15525 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:33:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559bf1501fd0 con 0x559bf1654110 2023-12-11T19:33:46.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.513+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf1654110 0x559bf1656530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49897fa700 10 client.15525 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49a5b9b700 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf1654110 0x559bf1656530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65612fb68983de7d server_cookie=ef5ab7691c178701 in_seq=0 out_seq=0 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49897fa700 1 -- 172.21.15.66:0/505800765 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559bf1501fd0 con 0x559bf1654110 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49897fa700 10 client.15525 handle_command_reply: tid=0 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49bb6d6b80 1 client.15525 shutdown 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 msgr2=0x559bf15f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.514+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 0x559bf15f4c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f499000b580 tx=0x7f4990005ef0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 shutdown_connections 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559bf1654110 0x559bf1656530 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559bf15e73d0 0x559bf15f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bf15e8c20 0x559bf15f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 --2- 172.21.15.66:0/505800765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559bf15e7d30 0x559bf15f4c80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 >> 172.21.15.66:0/505800765 conn(0x559bf15167e0 msgr2=0x559bf15ebb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49897fa700 0 client.15525 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 shutdown_connections 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 1 -- 172.21.15.66:0/505800765 wait complete. 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.515+0000 7f49bb6d6b80 20 client.15525 trim_cache size 0 max 16384 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:33:46.517 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: "69c12c31-b341-488c-94ae-23365c83e7ba": { 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "69c12c31-b341-488c-94ae-23365c83e7ba", 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:33:46.518 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f498c0a1480 msgr2=0x7f498c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f498c0a1480 0x7f498c0a3940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f499c009fd0 tx=0x7f499c009260 comp rx=0 tx=0).stop 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0101680 msgr2=0x7f49a0199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0101680 0x7f49a0199be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4990009fd0 tx=0x7f499000f690 comp rx=0 tx=0).stop 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 shutdown_connections 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f498c0a1480 0x7f498c0a3940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49a0102020 0x7f49a01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f49a0100490 0x7f49a01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 --2- 172.21.15.66:0/2968983299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49a0101680 0x7f49a0199be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 >> 172.21.15.66:0/2968983299 conn(0x7f49a00fba20 msgr2=0x7f49a0069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:46.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 shutdown_connections 2023-12-11T19:33:46.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:33:46.516+0000 7f49a7dff700 1 -- 172.21.15.66:0/2968983299 wait complete. 2023-12-11T19:33:46.535 INFO:tasks.cephfs.filesystem:scrub status for tag:69c12c31-b341-488c-94ae-23365c83e7ba - {'path': '/', 'tag': '69c12c31-b341-488c-94ae-23365c83e7ba', 'options': 'recursive,force'} 2023-12-11T19:33:47.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:46 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1789545585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:47.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:46 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1789545585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:47.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:46 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1789545585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:33:48.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:47 smithi167 ceph-mon[102750]: scrub queued for path: / 2023-12-11T19:33:48.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:47 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:33:48.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:47 smithi167 ceph-mon[102750]: scrub summary: active paths [/] 2023-12-11T19:33:48.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:47 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:33:48.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:47 smithi167 ceph-mon[102750]: pgmap v366: 97 pgs: 97 active+clean; 650 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-11T19:33:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:47 smithi066 ceph-mon[97920]: scrub queued for path: / 2023-12-11T19:33:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:47 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:33:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:47 smithi066 ceph-mon[97920]: scrub summary: active paths [/] 2023-12-11T19:33:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:47 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:33:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:47 smithi066 ceph-mon[97920]: pgmap v366: 97 pgs: 97 active+clean; 650 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-11T19:33:48.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:47 smithi070 ceph-mon[104044]: scrub queued for path: / 2023-12-11T19:33:48.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:47 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:33:48.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:47 smithi070 ceph-mon[104044]: scrub summary: active paths [/] 2023-12-11T19:33:48.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:47 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:33:48.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:47 smithi070 ceph-mon[104044]: pgmap v366: 97 pgs: 97 active+clean; 650 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-11T19:33:50.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:49 smithi167 ceph-mon[102750]: pgmap v367: 97 pgs: 97 active+clean; 650 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-12-11T19:33:50.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:49 smithi066 ceph-mon[97920]: pgmap v367: 97 pgs: 97 active+clean; 650 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-12-11T19:33:50.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:49 smithi070 ceph-mon[104044]: pgmap v367: 97 pgs: 97 active+clean; 650 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-12-11T19:33:52.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:51 smithi167 ceph-mon[102750]: pgmap v368: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-12-11T19:33:52.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:51 smithi167 ceph-mon[102750]: scrub summary: idle 2023-12-11T19:33:52.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:51 smithi066 ceph-mon[97920]: pgmap v368: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-12-11T19:33:52.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:51 smithi066 ceph-mon[97920]: scrub summary: idle 2023-12-11T19:33:52.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:51 smithi070 ceph-mon[104044]: pgmap v368: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-12-11T19:33:52.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:51 smithi070 ceph-mon[104044]: scrub summary: idle 2023-12-11T19:33:54.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:53 smithi167 ceph-mon[102750]: pgmap v369: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 9 op/s 2023-12-11T19:33:54.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:53 smithi066 ceph-mon[97920]: pgmap v369: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 9 op/s 2023-12-11T19:33:54.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:53 smithi070 ceph-mon[104044]: pgmap v369: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 9 op/s 2023-12-11T19:33:56.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:55 smithi167 ceph-mon[102750]: pgmap v370: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2023-12-11T19:33:56.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:55 smithi066 ceph-mon[97920]: pgmap v370: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2023-12-11T19:33:56.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:55 smithi070 ceph-mon[104044]: pgmap v370: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 10 op/s 2023-12-11T19:33:58.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:57 smithi167 ceph-mon[102750]: pgmap v371: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 10 op/s 2023-12-11T19:33:58.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:57 smithi066 ceph-mon[97920]: pgmap v371: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 10 op/s 2023-12-11T19:33:58.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:57 smithi070 ceph-mon[104044]: pgmap v371: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 10 op/s 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 -- 172.21.15.66:0/76607319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 msgr2=0x7f665c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 --2- 172.21.15.66:0/76607319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c109990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f665400b0d0 tx=0x7f665400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 -- 172.21.15.66:0/76607319 shutdown_connections 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 --2- 172.21.15.66:0/76607319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c109990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 --2- 172.21.15.66:0/76607319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c101550 0x7f665c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 --2- 172.21.15.66:0/76607319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c102740 0x7f665c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 -- 172.21.15.66:0/76607319 >> 172.21.15.66:0/76607319 conn(0x7f665c0fcac0 msgr2=0x7f665c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 -- 172.21.15.66:0/76607319 shutdown_connections 2023-12-11T19:33:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.192+0000 7f66611ff700 1 -- 172.21.15.66:0/76607319 wait complete. 2023-12-11T19:33:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.193+0000 7f66611ff700 1 Processor -- start 2023-12-11T19:33:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 -- start start 2023-12-11T19:33:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c101550 0x7f665c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c102740 0x7f665c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a1200 con 0x7f665c1030e0 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a1340 con 0x7f665c101550 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f66611ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1a14d0 con 0x7f665c102740 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f665b59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c101550 0x7f665c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f6653fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c102740 0x7f665c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c101550 0x7f665c1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36812/0 (socket says 172.21.15.66:36812) 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.194+0000 7f665ad9d700 1 -- 172.21.15.66:0/429254866 learned_addr learned my addr 172.21.15.66:0/429254866 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f665b59e700 1 -- 172.21.15.66:0/429254866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c102740 msgr2=0x7f665c199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f665b59e700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c102740 0x7f665c199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f665b59e700 1 -- 172.21.15.66:0/429254866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c101550 msgr2=0x7f665c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f665b59e700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c101550 0x7f665c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f665b59e700 1 -- 172.21.15.66:0/429254866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009d00 con 0x7f665c1030e0 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f665b59e700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c1a0c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6654007b60 tx=0x7f6654009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f6658d99700 1 -- 172.21.15.66:0/429254866 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400e040 con 0x7f665c1030e0 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c1a1750 con 0x7f665c1030e0 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f6658d99700 1 -- 172.21.15.66:0/429254866 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6654003e10 con 0x7f665c1030e0 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f6658d99700 1 -- 172.21.15.66:0/429254866 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665401c8c0 con 0x7f665c1030e0 2023-12-11T19:33:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.195+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c1a1b80 con 0x7f665c1030e0 2023-12-11T19:33:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.199+0000 7f6658d99700 1 -- 172.21.15.66:0/429254866 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6654005690 con 0x7f665c1030e0 2023-12-11T19:33:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.200+0000 7f6674adeb80 1 Processor -- start 2023-12-11T19:33:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.200+0000 7f6658d99700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66480a1480 0x7f66480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.200+0000 7f6674adeb80 1 -- start start 2023-12-11T19:33:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.200+0000 7f6658d99700 1 -- 172.21.15.66:0/429254866 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66540cdbf0 con 0x7f665c1030e0 2023-12-11T19:33:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.200+0000 7f6653fff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66480a1480 0x7f66480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64ced0 0x5648be64d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be64e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 0x5648be64eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6674adeb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be5a8af0 con 0x5648be64e720 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6674adeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be5a6410 con 0x5648be64ced0 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6674adeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be5a1d00 con 0x5648be64d830 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 0x5648be64eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64ced0 0x5648be64d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 0x5648be64eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37372/0 (socket says 172.21.15.66:37372) 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 -- 172.21.15.66:0/2011476066 learned_addr learned my addr 172.21.15.66:0/2011476066 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6653fff700 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be64e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f6653fff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66480a1480 0x7f66480a3940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6644005950 tx=0x7f66440106b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 -- 172.21.15.66:0/2011476066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 msgr2=0x5648be64e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be64e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 -- 172.21.15.66:0/2011476066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64ced0 msgr2=0x5648be64d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64ced0 0x5648be64d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.201+0000 7f665b59e700 1 -- 172.21.15.66:0/2011476066 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648be41e5d0 con 0x5648be64e720 2023-12-11T19:33:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.202+0000 7f665b59e700 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 0x5648be64eba0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f665409af80 tx=0x7f665400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23596f18fbf530 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.202+0000 7f66517fa700 1 -- 172.21.15.66:0/2011476066 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665409b6f0 con 0x5648be64e720 2023-12-11T19:33:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.202+0000 7f66517fa700 1 -- 172.21.15.66:0/2011476066 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f665409aff0 con 0x5648be64e720 2023-12-11T19:33:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.202+0000 7f66517fa700 1 -- 172.21.15.66:0/2011476066 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540a1070 con 0x5648be64e720 2023-12-11T19:33:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 -- 172.21.15.66:0/2011476066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 msgr2=0x5648be64eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 0x5648be64eba0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f665409af80 tx=0x7f665400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 -- 172.21.15.66:0/2011476066 shutdown_connections 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64ced0 0x5648be64d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64e720 0x5648be64eba0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2011476066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be64e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 -- 172.21.15.66:0/2011476066 >> 172.21.15.66:0/2011476066 conn(0x5648be57c780 msgr2=0x5648be649880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 -- 172.21.15.66:0/2011476066 shutdown_connections 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 -- 172.21.15.66:0/2011476066 wait complete. 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.203+0000 7f6674adeb80 1 -- start start 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 Processor -- start 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be656b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be657040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be65de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be55b500 con 0x5648be64ced0 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be55b1a0 con 0x5648be64d830 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be4208c0 con 0x5648be64e720 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6674adeb80 10 client.0 _ll_register_callbacks cb 0x5648be486a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be656b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be656b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37374/0 (socket says 172.21.15.66:37374) 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f6653fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be657040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.204+0000 7f665ad9d700 1 -- 172.21.15.66:0/574079657 learned_addr learned my addr 172.21.15.66:0/574079657 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665b59e700 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be65de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665ad9d700 1 -- 172.21.15.66:0/574079657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 msgr2=0x5648be65de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665ad9d700 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be65de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665ad9d700 1 -- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 msgr2=0x5648be657040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665ad9d700 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be657040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665ad9d700 1 -- 172.21.15.66:0/574079657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c1a1b80 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f665ad9d700 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be656b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f664c0116f0 tx=0x7f664c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee83398b186489f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f66427fc700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f66427fc700 1 -- 172.21.15.66:0/574079657 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c016070 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f66427fc700 1 -- 172.21.15.66:0/574079657 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f664c010640 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.205+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648be640730 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f66427fc700 1 -- 172.21.15.66:0/574079657 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664c010800 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 10 client.15537 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648be3cad80 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f66427fc700 1 -- 172.21.15.66:0/574079657 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5648be640730 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f66427fc700 1 -- 172.21.15.66:0/574079657 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f664c01bab0 con 0x5648be64ced0 2023-12-11T19:33:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 20 client.15537 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be662900 0x5648be664d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 4 client.15537 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.206+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648be61cee0 con 0x5648be662900 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.207+0000 7f6653fff700 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be662900 0x5648be664d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.207+0000 7f66427fc700 10 client.15537 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.207+0000 7f6653fff700 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be662900 0x5648be664d20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a99d92fea9cb4d0 server_cookie=dadccceac0b2edeb in_seq=0 out_seq=0 2023-12-11T19:33:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f66427fc700 1 -- 172.21.15.66:0/574079657 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648be61cee0 con 0x5648be662900 2023-12-11T19:33:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f66427fc700 10 client.15537 handle_command_reply: tid=0 2023-12-11T19:33:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 client.15537 shutdown 2023-12-11T19:33:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 msgr2=0x5648be656b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be656b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f664c0116f0 tx=0x7f664c011a00 comp rx=0 tx=0).stop 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 shutdown_connections 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be662900 0x5648be664d20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be65de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f66427fc700 0 client.15537 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be656b00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 --2- 172.21.15.66:0/574079657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be657040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.208+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 >> 172.21.15.66:0/574079657 conn(0x5648be57c780 msgr2=0x5648be647ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.209+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 shutdown_connections 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.209+0000 7f6674adeb80 1 -- 172.21.15.66:0/574079657 wait complete. 2023-12-11T19:33:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.209+0000 7f6674adeb80 20 client.15537 trim_cache size 0 max 16384 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.224+0000 7f6674adeb80 1 Processor -- start 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.224+0000 7f6674adeb80 1 -- start start 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be64eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be65de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be6568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6674adeb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be640700 con 0x5648be64ced0 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6674adeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be65e470 con 0x5648be64d830 2023-12-11T19:33:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6674adeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be669f50 con 0x5648be64e720 2023-12-11T19:33:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be64eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be64eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37390/0 (socket says 172.21.15.66:37390) 2023-12-11T19:33:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 -- 172.21.15.66:0/1368045741 learned_addr learned my addr 172.21.15.66:0/1368045741 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f6653fff700 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be65de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665b59e700 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be6568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 -- 172.21.15.66:0/1368045741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 msgr2=0x5648be6568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be6568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 -- 172.21.15.66:0/1368045741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 msgr2=0x5648be65de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.225+0000 7f665ad9d700 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be65de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.226+0000 7f665ad9d700 1 -- 172.21.15.66:0/1368045741 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648be3b3c00 con 0x5648be64ced0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.226+0000 7f665ad9d700 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be64eb60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f664c0116c0 tx=0x7f664c005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7126804b68c974a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.226+0000 7f6651ffb700 1 -- 172.21.15.66:0/1368045741 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c016040 con 0x5648be64ced0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.226+0000 7f6651ffb700 1 -- 172.21.15.66:0/1368045741 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f664c0054c0 con 0x5648be64ced0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.226+0000 7f6651ffb700 1 -- 172.21.15.66:0/1368045741 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c025e60 con 0x5648be64ced0 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 -- 172.21.15.66:0/1368045741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 msgr2=0x5648be64eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be64eb60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f664c0116c0 tx=0x7f664c005c20 comp rx=0 tx=0).stop 2023-12-11T19:33:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 -- 172.21.15.66:0/1368045741 shutdown_connections 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64e720 0x5648be6568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be64eb60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 --2- 172.21.15.66:0/1368045741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64d830 0x5648be65de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 -- 172.21.15.66:0/1368045741 >> 172.21.15.66:0/1368045741 conn(0x5648be57c780 msgr2=0x5648be649460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 -- 172.21.15.66:0/1368045741 shutdown_connections 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 -- 172.21.15.66:0/1368045741 wait complete. 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 -- start start 2023-12-11T19:33:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.227+0000 7f6674adeb80 1 Processor -- start 2023-12-11T19:33:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be65c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be65c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64e720 0x5648be65d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be65d670 con 0x5648be64ced0 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be65d7e0 con 0x5648be64e720 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648be65d9a0 con 0x5648be64d830 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6674adeb80 10 client.0 _ll_register_callbacks cb 0x5648be486a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be65c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f665ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be65c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37394/0 (socket says 172.21.15.66:37394) 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f665ad9d700 1 -- 172.21.15.66:0/2994960873 learned_addr learned my addr 172.21.15.66:0/2994960873 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.228+0000 7f6653fff700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be65c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665b59e700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64e720 0x5648be65d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665ad9d700 1 -- 172.21.15.66:0/2994960873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 msgr2=0x5648be65c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665ad9d700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be65c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665ad9d700 1 -- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64e720 msgr2=0x5648be65d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665ad9d700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64e720 0x5648be65d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665ad9d700 1 -- 172.21.15.66:0/2994960873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648be3cad80 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f665ad9d700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be65c300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f664c0116f0 tx=0x7f664c00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54c6148dddff5cb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f66427fc700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f66427fc700 1 -- 172.21.15.66:0/2994960873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c016040 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f665409aff0 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f66427fc700 1 -- 172.21.15.66:0/2994960873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f664c021030 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.229+0000 7f66427fc700 1 -- 172.21.15.66:0/2994960873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664c025d70 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 10 client.15543 fetch_fsmap learned FSMap version 27 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648be3f11f0 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f66427fc700 1 -- 172.21.15.66:0/2994960873 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f665409aff0 con 0x5648be64ced0 2023-12-11T19:33:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f66427fc700 1 -- 172.21.15.66:0/2994960873 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f664c0107e0 con 0x5648be64ced0 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 20 client.15543 populate_metadata read hostname 'smithi066' 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be6b9b80 0x5648be6bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 4 client.15543 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648be567f70 con 0x5648be6b9b80 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.230+0000 7f6653fff700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be6b9b80 0x5648be6bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.231+0000 7f6653fff700 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be6b9b80 0x5648be6bbfa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12c581147e5e105 server_cookie=e929f1055c793d65 in_seq=0 out_seq=0 2023-12-11T19:33:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.231+0000 7f66427fc700 10 client.15543 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.243+0000 7f66427fc700 1 -- 172.21.15.66:0/2994960873 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648be567f70 con 0x5648be6b9b80 2023-12-11T19:33:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.243+0000 7f66427fc700 10 client.15543 handle_command_reply: tid=0 2023-12-11T19:33:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.243+0000 7f6674adeb80 1 client.15543 shutdown 2023-12-11T19:33:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.243+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 msgr2=0x5648be65c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.243+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be65c300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f664c0116f0 tx=0x7f664c00b9c0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 shutdown_connections 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5648be6b9b80 0x5648be6bbfa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5648be64e720 0x5648be65d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648be64ced0 0x5648be65c300 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f66427fc700 0 client.15543 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 --2- 172.21.15.66:0/2994960873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648be64d830 0x5648be65c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 >> 172.21.15.66:0/2994960873 conn(0x5648be57c780 msgr2=0x5648be651370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 shutdown_connections 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 1 -- 172.21.15.66:0/2994960873 wait complete. 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f6674adeb80 20 client.15543 trim_cache size 0 max 16384 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:33:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:33:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:33:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:33:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66480a1480 msgr2=0x7f66480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f66611ff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66480a1480 0x7f66480a3940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6644005950 tx=0x7f66440106b0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 msgr2=0x7f665c1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.244+0000 7f66611ff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c1a0c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6654007b60 tx=0x7f6654009440 comp rx=0 tx=0).stop 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 shutdown_connections 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f66480a1480 0x7f66480a3940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f665c101550 0x7f665c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f665c1030e0 0x7f665c1a0c30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 --2- 172.21.15.66:0/429254866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c102740 0x7f665c199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 >> 172.21.15.66:0/429254866 conn(0x7f665c0fcac0 msgr2=0x7f665c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:33:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 shutdown_connections 2023-12-11T19:33:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:33:59.245+0000 7f66611ff700 1 -- 172.21.15.66:0/429254866 wait complete. 2023-12-11T19:34:00.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:33:59 smithi167 ceph-mon[102750]: pgmap v372: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-11T19:34:00.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:33:59 smithi066 ceph-mon[97920]: pgmap v372: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-11T19:34:00.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:33:59 smithi070 ceph-mon[104044]: pgmap v372: 97 pgs: 97 active+clean; 655 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-11T19:34:02.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:01 smithi167 ceph-mon[102750]: pgmap v373: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-11T19:34:02.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:01 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:34:02.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:01 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:02.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:01 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:01 smithi066 ceph-mon[97920]: pgmap v373: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-11T19:34:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:01 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:34:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:01 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:01 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:02.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:01 smithi070 ceph-mon[104044]: pgmap v373: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-11T19:34:02.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:01 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:34:02.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:01 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:02.360 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:01 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:04.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:03 smithi167 ceph-mon[102750]: pgmap v374: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-11T19:34:04.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:03 smithi066 ceph-mon[97920]: pgmap v374: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-11T19:34:04.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:03 smithi070 ceph-mon[104044]: pgmap v374: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-11T19:34:06.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: pgmap v375: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2023-12-11T19:34:06.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:05 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: pgmap v375: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2023-12-11T19:34:06.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:05 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: pgmap v375: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2023-12-11T19:34:06.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:06.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:05 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:07.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:34:07.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:34:07.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:06 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:07.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:34:07.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:34:07.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:06 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:07.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:34:07.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:34:07.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:06 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:34:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:07 smithi167 ceph-mon[102750]: pgmap v376: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-12-11T19:34:08.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:07 smithi066 ceph-mon[97920]: pgmap v376: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-12-11T19:34:08.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:07 smithi070 ceph-mon[104044]: pgmap v376: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-12-11T19:34:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:09 smithi167 ceph-mon[102750]: pgmap v377: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2023-12-11T19:34:10.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:09 smithi066 ceph-mon[97920]: pgmap v377: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2023-12-11T19:34:10.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:09 smithi070 ceph-mon[104044]: pgmap v377: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2023-12-11T19:34:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:11 smithi167 ceph-mon[102750]: pgmap v378: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-12-11T19:34:12.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:11 smithi066 ceph-mon[97920]: pgmap v378: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-12-11T19:34:12.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:11 smithi070 ceph-mon[104044]: pgmap v378: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-12-11T19:34:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.410+0000 7f25339ff700 1 -- 172.21.15.66:0/489533811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c074030 msgr2=0x7f252c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.410+0000 7f25339ff700 1 --2- 172.21.15.66:0/489533811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c074030 0x7f252c0744b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f251c009990 tx=0x7f251c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 -- 172.21.15.66:0/489533811 shutdown_connections 2023-12-11T19:34:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 --2- 172.21.15.66:0/489533811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 0x7f252c073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 --2- 172.21.15.66:0/489533811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f252c10db00 0x7f252c10fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 --2- 172.21.15.66:0/489533811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c074030 0x7f252c0744b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 -- 172.21.15.66:0/489533811 >> 172.21.15.66:0/489533811 conn(0x7f252c0fcaa0 msgr2=0x7f252c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 -- 172.21.15.66:0/489533811 shutdown_connections 2023-12-11T19:34:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.411+0000 7f25339ff700 1 -- 172.21.15.66:0/489533811 wait complete. 2023-12-11T19:34:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 Processor -- start 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 -- start start 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f252c074030 0x7f252c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 0x7f252c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 0x7f252c1a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c1a5520 con 0x7f252c10db00 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c1a5660 con 0x7f252c075610 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.412+0000 7f25339ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c1a57f0 con 0x7f252c074030 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 0x7f252c1a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2530f9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 0x7f252c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 0x7f252c1a4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48356/0 (socket says 172.21.15.66:48356) 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 -- 172.21.15.66:0/446657037 learned_addr learned my addr 172.21.15.66:0/446657037 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f253179b700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f252c074030 0x7f252c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 -- 172.21.15.66:0/446657037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f252c074030 msgr2=0x7f252c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f252c074030 0x7f252c19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 -- 172.21.15.66:0/446657037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 msgr2=0x7f252c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 0x7f252c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 -- 172.21.15.66:0/446657037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c0095f0 con 0x7f252c10db00 2023-12-11T19:34:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2530f9a700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 0x7f252c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.413+0000 7f2531f9c700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 0x7f252c1a4f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f252400ef30 tx=0x7f252400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.414+0000 7f25227fc700 1 -- 172.21.15.66:0/446657037 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524018070 con 0x7f252c10db00 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.414+0000 7f25227fc700 1 -- 172.21.15.66:0/446657037 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f252400f040 con 0x7f252c10db00 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.414+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c1a5a70 con 0x7f252c10db00 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.414+0000 7f25227fc700 1 -- 172.21.15.66:0/446657037 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400cbc0 con 0x7f252c10db00 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.414+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c1a5f80 con 0x7f252c10db00 2023-12-11T19:34:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.417+0000 7f25227fc700 1 -- 172.21.15.66:0/446657037 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f252400cd80 con 0x7f252c10db00 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f2547227b80 1 Processor -- start 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f2547227b80 1 -- start start 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f25227fc700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25180a1500 0x7f25180a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f25227fc700 1 -- 172.21.15.66:0/446657037 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25240cc7a0 con 0x7f252c10db00 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f2547227b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d7b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f2547227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7b830 0x563567d7c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f2547227b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7c720 0x563567d7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.418+0000 7f2547227b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567cd6af0 con 0x563567d7aed0 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f2547227b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567cd4410 con 0x563567d7c720 2023-12-11T19:34:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f2547227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567ccfd00 con 0x563567d7b830 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f2530f9a700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25180a1500 0x7f25180a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d7b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d7b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48366/0 (socket says 172.21.15.66:48366) 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 -- 172.21.15.66:0/32104425 learned_addr learned my addr 172.21.15.66:0/32104425 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f2530f9a700 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7b830 0x563567d7c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f2531f9c700 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7c720 0x563567d7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 -- 172.21.15.66:0/32104425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7b830 msgr2=0x563567d7c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7b830 0x563567d7c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 -- 172.21.15.66:0/32104425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7c720 msgr2=0x563567d7cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7c720 0x563567d7cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.419+0000 7f253179b700 1 -- 172.21.15.66:0/32104425 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563567b4c5d0 con 0x563567d7aed0 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.420+0000 7f253179b700 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d7b2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2528009d80 tx=0x7f2528011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9ea33fc7698cd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.420+0000 7f2530f9a700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25180a1500 0x7f25180a39c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f251c006010 tx=0x7f251c004040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.420+0000 7f25177fe700 1 -- 172.21.15.66:0/32104425 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528004230 con 0x563567d7aed0 2023-12-11T19:34:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.420+0000 7f25177fe700 1 -- 172.21.15.66:0/32104425 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2528004390 con 0x563567d7aed0 2023-12-11T19:34:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.420+0000 7f25177fe700 1 -- 172.21.15.66:0/32104425 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528019730 con 0x563567d7aed0 2023-12-11T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 -- 172.21.15.66:0/32104425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 msgr2=0x563567d7b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d7b2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2528009d80 tx=0x7f2528011720 comp rx=0 tx=0).stop 2023-12-11T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 -- 172.21.15.66:0/32104425 shutdown_connections 2023-12-11T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7c720 0x563567d7cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d7b2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 --2- 172.21.15.66:0/32104425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7b830 0x563567d7c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 -- 172.21.15.66:0/32104425 >> 172.21.15.66:0/32104425 conn(0x563567caa780 msgr2=0x563567d77880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 -- 172.21.15.66:0/32104425 shutdown_connections 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 -- 172.21.15.66:0/32104425 wait complete. 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.421+0000 7f2547227b80 1 -- start start 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 Processor -- start 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d84df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7b830 0x563567d85330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567c89500 con 0x563567d7aed0 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567c891a0 con 0x563567d7b830 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567b4e8c0 con 0x563567d7c720 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2547227b80 10 client.0 _ll_register_callbacks cb 0x563567bb4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f253179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d84df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f253179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d84df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48372/0 (socket says 172.21.15.66:48372) 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f253179b700 1 -- 172.21.15.66:0/1230813282 learned_addr learned my addr 172.21.15.66:0/1230813282 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2530f9a700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7b830 0x563567d85330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.422+0000 7f2531f9c700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f253179b700 1 -- 172.21.15.66:0/1230813282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 msgr2=0x563567d8c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f253179b700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f253179b700 1 -- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7b830 msgr2=0x563567d85330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f253179b700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7b830 0x563567d85330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f253179b700 1 -- 172.21.15.66:0/1230813282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25280099e0 con 0x563567d7aed0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f2531f9c700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f253179b700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d84df0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2528000c00 tx=0x7f2528003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2645bc82bffd8a server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f2514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f2514ff9700 1 -- 172.21.15.66:0/1230813282 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528005ce0 con 0x563567d7aed0 2023-12-11T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563567d6e730 con 0x563567d7aed0 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.423+0000 7f2514ff9700 1 -- 172.21.15.66:0/1230813282 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2528005250 con 0x563567d7aed0 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2514ff9700 1 -- 172.21.15.66:0/1230813282 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2528005410 con 0x563567d7aed0 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 10 client.15555 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563567af8d80 con 0x563567d7aed0 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2514ff9700 1 -- 172.21.15.66:0/1230813282 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563567d6e730 con 0x563567d7aed0 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2514ff9700 1 -- 172.21.15.66:0/1230813282 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2528021040 con 0x563567d7aed0 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 20 client.15555 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567d90b90 0x563567d92fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 4 client.15555 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.424+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563567d4aee0 con 0x563567d90b90 2023-12-11T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.425+0000 7f2530f9a700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567d90b90 0x563567d92fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.426+0000 7f2530f9a700 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567d90b90 0x563567d92fb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c779ff2d17760b server_cookie=718bbfeb559cb94a in_seq=0 out_seq=0 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.426+0000 7f2514ff9700 10 client.15555 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2514ff9700 1 -- 172.21.15.66:0/1230813282 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563567d4aee0 con 0x563567d90b90 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2514ff9700 10 client.15555 handle_command_reply: tid=0 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 client.15555 shutdown 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 msgr2=0x563567d84df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d84df0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2528000c00 tx=0x7f2528003930 comp rx=0 tx=0).stop 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 shutdown_connections 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567d90b90 0x563567d92fb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7aed0 0x563567d84df0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2514ff9700 0 client.15555 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 --2- 172.21.15.66:0/1230813282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7b830 0x563567d85330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 >> 172.21.15.66:0/1230813282 conn(0x563567caa780 msgr2=0x563567d75ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 shutdown_connections 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 1 -- 172.21.15.66:0/1230813282 wait complete. 2023-12-11T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.427+0000 7f2547227b80 20 client.15555 trim_cache size 0 max 16384 2023-12-11T19:34:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 Processor -- start 2023-12-11T19:34:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 -- start start 2023-12-11T19:34:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d82b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567d8c760 con 0x563567d7b830 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567d98200 con 0x563567d7aed0 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2547227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567d8ce00 con 0x563567d7c720 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.443+0000 7f2530f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48376/0 (socket says 172.21.15.66:48376) 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 -- 172.21.15.66:0/1995144986 learned_addr learned my addr 172.21.15.66:0/1995144986 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2531f9c700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d82b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f253179b700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 -- 172.21.15.66:0/1995144986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 msgr2=0x563567d82b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d82b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 -- 172.21.15.66:0/1995144986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 msgr2=0x563567d7cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d7cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 -- 172.21.15.66:0/1995144986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563567ae1c00 con 0x563567d7b830 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f253179b700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d7cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2531f9c700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d82b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.444+0000 7f2530f9a700 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8c120 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f251c009990 tx=0x7f251c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60698f66e2e69244 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2517fff700 1 -- 172.21.15.66:0/1995144986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c01d040 con 0x563567d7b830 2023-12-11T19:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2517fff700 1 -- 172.21.15.66:0/1995144986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f251c0343c0 con 0x563567d7b830 2023-12-11T19:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2517fff700 1 -- 172.21.15.66:0/1995144986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c004760 con 0x563567d7b830 2023-12-11T19:34:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 -- 172.21.15.66:0/1995144986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 msgr2=0x563567d8c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8c120 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f251c009990 tx=0x7f251c01a050 comp rx=0 tx=0).stop 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 -- 172.21.15.66:0/1995144986 shutdown_connections 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d7cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d82b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 --2- 172.21.15.66:0/1995144986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8c120 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 -- 172.21.15.66:0/1995144986 >> 172.21.15.66:0/1995144986 conn(0x563567caa780 msgr2=0x563567d840f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 -- 172.21.15.66:0/1995144986 shutdown_connections 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.445+0000 7f2547227b80 1 -- 172.21.15.66:0/1995144986 wait complete. 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 -- start start 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 Processor -- start 2023-12-11T19:34:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d89bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567d8b580 con 0x563567d7b830 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567d8b6f0 con 0x563567d7aed0 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563567d8b8b0 con 0x563567d7c720 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.446+0000 7f2547227b80 10 client.0 _ll_register_callbacks cb 0x563567bb4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48380/0 (socket says 172.21.15.66:48380) 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 -- 172.21.15.66:0/3614425010 learned_addr learned my addr 172.21.15.66:0/3614425010 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2531f9c700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f253179b700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d89bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 -- 172.21.15.66:0/3614425010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 msgr2=0x563567d8b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 -- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 msgr2=0x563567d89bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d89bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 -- 172.21.15.66:0/3614425010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c1a5f80 con 0x563567d7b830 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f253179b700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2530f9a700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8a130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b650 tx=0x7f251c00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef276be2cd9e2011 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.447+0000 7f2514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2514ff9700 1 -- 172.21.15.66:0/3614425010 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c01d040 con 0x563567d7b830 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2528004390 con 0x563567d7b830 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2514ff9700 1 -- 172.21.15.66:0/3614425010 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f251c034530 con 0x563567d7b830 2023-12-11T19:34:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2514ff9700 1 -- 172.21.15.66:0/3614425010 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f251c0346f0 con 0x563567d7b830 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 10 client.15561 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f251c020040 con 0x563567d7b830 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2514ff9700 1 -- 172.21.15.66:0/3614425010 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2528004390 con 0x563567d7b830 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2514ff9700 1 -- 172.21.15.66:0/3614425010 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f251c033040 con 0x563567d7b830 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 20 client.15561 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567de78c0 0x563567de9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 4 client.15561 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.448+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563567c95f70 con 0x563567de78c0 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.449+0000 7f253179b700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567de78c0 0x563567de9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.449+0000 7f253179b700 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567de78c0 0x563567de9cb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe2c774c3a0732 server_cookie=ed5757ca4d2fc5a0 in_seq=0 out_seq=0 2023-12-11T19:34:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.449+0000 7f2514ff9700 10 client.15561 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2514ff9700 1 -- 172.21.15.66:0/3614425010 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563567c95f70 con 0x563567de78c0 2023-12-11T19:34:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2514ff9700 10 client.15561 handle_command_reply: tid=0 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 client.15561 shutdown 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 msgr2=0x563567d8a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8a130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b650 tx=0x7f251c00bbb0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 shutdown_connections 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x563567de78c0 0x563567de9cb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563567d7aed0 0x563567d89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563567d7c720 0x563567d8b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2514ff9700 0 client.15561 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 --2- 172.21.15.66:0/3614425010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563567d7b830 0x563567d8a130 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.453+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 >> 172.21.15.66:0/3614425010 conn(0x563567caa780 msgr2=0x563567d77460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 shutdown_connections 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f2547227b80 1 -- 172.21.15.66:0/3614425010 wait complete. 2023-12-11T19:34:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f2547227b80 20 client.15561 trim_cache size 0 max 16384 2023-12-11T19:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:34:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25180a1500 msgr2=0x7f25180a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25180a1500 0x7f25180a39c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f251c006010 tx=0x7f251c004040 comp rx=0 tx=0).stop 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 msgr2=0x7f252c1a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 0x7f252c1a4f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f252400ef30 tx=0x7f252400c3a0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 shutdown_connections 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25180a1500 0x7f25180a39c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f252c10db00 0x7f252c1a4f50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f252c074030 0x7f252c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 --2- 172.21.15.66:0/446657037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c075610 0x7f252c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.454+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 >> 172.21.15.66:0/446657037 conn(0x7f252c0fcaa0 msgr2=0x7f252c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.455+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 shutdown_connections 2023-12-11T19:34:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:12.455+0000 7f25339ff700 1 -- 172.21.15.66:0/446657037 wait complete. 2023-12-11T19:34:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:13 smithi167 ceph-mon[102750]: pgmap v379: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 12 op/s 2023-12-11T19:34:14.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:13 smithi066 ceph-mon[97920]: pgmap v379: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 12 op/s 2023-12-11T19:34:14.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:13 smithi070 ceph-mon[104044]: pgmap v379: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 12 op/s 2023-12-11T19:34:16.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:15 smithi167 ceph-mon[102750]: pgmap v380: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 12 op/s 2023-12-11T19:34:16.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:15 smithi066 ceph-mon[97920]: pgmap v380: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 12 op/s 2023-12-11T19:34:16.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:15 smithi070 ceph-mon[104044]: pgmap v380: 97 pgs: 97 active+clean; 661 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 12 op/s 2023-12-11T19:34:16.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:34:16.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 -- 172.21.15.66:0/2093595318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 msgr2=0x7fd2ec1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/2093595318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec1019c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0099c0 tx=0x7fd2dc009cd0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 -- 172.21.15.66:0/2093595318 shutdown_connections 2023-12-11T19:34:16.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/2093595318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 0x7fd2ec1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/2093595318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec1019c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/2093595318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec102790 0x7fd2ec102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 -- 172.21.15.66:0/2093595318 >> 172.21.15.66:0/2093595318 conn(0x7fd2ec0fcb10 msgr2=0x7fd2ec0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:16.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 -- 172.21.15.66:0/2093595318 shutdown_connections 2023-12-11T19:34:16.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.769+0000 7fd2f13ff700 1 -- 172.21.15.66:0/2093595318 wait complete. 2023-12-11T19:34:16.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.770+0000 7fd2f13ff700 1 Processor -- start 2023-12-11T19:34:16.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.770+0000 7fd2f13ff700 1 -- start start 2023-12-11T19:34:16.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2f13ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2f13ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec102790 0x7fd2ec0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2f13ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 0x7fd2ec073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2f13ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec0746a0 con 0x7fd2ec1015a0 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2f13ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec074810 con 0x7fd2ec102790 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2f13ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1a1510 con 0x7fd2ec103130 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2eaffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2eaffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48398/0 (socket says 172.21.15.66:48398) 2023-12-11T19:34:16.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2eaffd700 1 -- 172.21.15.66:0/627478421 learned_addr learned my addr 172.21.15.66:0/627478421 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec102790 0x7fd2ec0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.771+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 0x7fd2ec073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eaffd700 1 -- 172.21.15.66:0/627478421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 msgr2=0x7fd2ec073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 0x7fd2ec073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eaffd700 1 -- 172.21.15.66:0/627478421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec102790 msgr2=0x7fd2ec0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec102790 0x7fd2ec0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eaffd700 1 -- 172.21.15.66:0/627478421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0095f0 con 0x7fd2ec1015a0 2023-12-11T19:34:16.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 0x7fd2ec073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:16.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec075330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009f50 tx=0x7fd2dc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:16.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2d3fff700 1 -- 172.21.15.66:0/627478421 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc01d070 con 0x7fd2ec1015a0 2023-12-11T19:34:16.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.772+0000 7fd2d3fff700 1 -- 172.21.15.66:0/627478421 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2dc00bda0 con 0x7fd2ec1015a0 2023-12-11T19:34:16.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.773+0000 7fd2d3fff700 1 -- 172.21.15.66:0/627478421 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc022dd0 con 0x7fd2ec1015a0 2023-12-11T19:34:16.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.773+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1a1740 con 0x7fd2ec1015a0 2023-12-11T19:34:16.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.773+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1a1ab0 con 0x7fd2ec1015a0 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.778+0000 7fd2d3fff700 1 -- 172.21.15.66:0/627478421 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd2dc022570 con 0x7fd2ec1015a0 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 Processor -- start 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 -- start start 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd2d3fff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d40a1480 0x7fd2d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd2d3fff700 1 -- 172.21.15.66:0/627478421 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2dc0ce050 con 0x7fd2ec1015a0 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e29640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0d83b50 con 0x5569d0e28d30 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0d81470 con 0x5569d0e29c20 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.779+0000 7fd304df8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0d7cd60 con 0x5569d0e283d0 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d40a1480 0x7fd2d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2ea7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e287f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:49502/0 (socket says 172.21.15.66:49502) 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 -- 172.21.15.66:0/2517500386 learned_addr learned my addr 172.21.15.66:0/2517500386 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e2a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:51544/0 (socket says 172.21.15.66:51544) 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 -- 172.21.15.66:0/2517500386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 msgr2=0x5569d0e2a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e2a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 -- 172.21.15.66:0/2517500386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 msgr2=0x5569d0e29640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e29640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eaffd700 1 -- 172.21.15.66:0/2517500386 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d0bf9630 con 0x5569d0e283d0 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e2a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.780+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e29640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.781+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d40a1480 0x7fd2d40a3940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009fd0 tx=0x7fd2e4009c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.781+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e287f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0a3210 tx=0x7fd2dc009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43caa8764d00bdc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.781+0000 7fd2d17fa700 1 -- 172.21.15.66:0/2517500386 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc09bb40 con 0x5569d0e283d0 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.781+0000 7fd2d17fa700 1 -- 172.21.15.66:0/2517500386 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2dc09b440 con 0x5569d0e283d0 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.781+0000 7fd2d17fa700 1 -- 172.21.15.66:0/2517500386 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc09f380 con 0x5569d0e283d0 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 -- 172.21.15.66:0/2517500386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 msgr2=0x5569d0e287f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e287f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0a3210 tx=0x7fd2dc009cd0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 -- 172.21.15.66:0/2517500386 shutdown_connections 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e2a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e287f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 --2- 172.21.15.66:0/2517500386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e29640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 -- 172.21.15.66:0/2517500386 >> 172.21.15.66:0/2517500386 conn(0x5569d0d577e0 msgr2=0x5569d0e24d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:16.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 -- 172.21.15.66:0/2517500386 shutdown_connections 2023-12-11T19:34:16.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.782+0000 7fd304df8b80 1 -- 172.21.15.66:0/2517500386 wait complete. 2023-12-11T19:34:16.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 -- start start 2023-12-11T19:34:16.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 Processor -- start 2023-12-11T19:34:16.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e31ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e32400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0d36560 con 0x5569d0e28d30 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0d36200 con 0x5569d0e29c20 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.783+0000 7fd304df8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0bfb920 con 0x5569d0e283d0 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd304df8b80 10 client.0 _ll_register_callbacks cb 0x5569d0c61a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e32400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e32400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48420/0 (socket says 172.21.15.66:48420) 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 -- 172.21.15.66:0/3514756929 learned_addr learned my addr 172.21.15.66:0/3514756929 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e31ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 -- 172.21.15.66:0/3514756929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 msgr2=0x5569d0e31ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e31ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 -- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 msgr2=0x5569d0e39440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e39440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2ea7fc700 1 -- 172.21.15.66:0/3514756929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e4017740 con 0x5569d0e28d30 2023-12-11T19:34:16.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.784+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e39440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e32400 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000b960 tx=0x7fd2e000bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87683e5b7c44103 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd2ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd2ceff5700 1 -- 172.21.15.66:0/3514756929 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0016070 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd2ceff5700 1 -- 172.21.15.66:0/3514756929 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2e0003d00 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569d0e1bb40 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd2ceff5700 1 -- 172.21.15.66:0/3514756929 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2e0011440 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd304df8b80 10 client.15573 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd2ec1a1ab0 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.785+0000 7fd2ceff5700 1 -- 172.21.15.66:0/3514756929 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5569d0e1bb40 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd2ceff5700 1 -- 172.21.15.66:0/3514756929 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2e001b540 con 0x5569d0e28d30 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd304df8b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd304df8b80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd304df8b80 20 client.15573 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e3dd60 0x5569d0e40180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd304df8b80 4 client.15573 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:16.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569d0df7f40 con 0x5569d0e3dd60 2023-12-11T19:34:16.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.786+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e3dd60 0x5569d0e40180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.787+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e3dd60 0x5569d0e40180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d0845757ff6f35 server_cookie=170f448bfa7dcdc6 in_seq=0 out_seq=0 2023-12-11T19:34:16.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.787+0000 7fd2ceff5700 10 client.15573 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd2ceff5700 1 -- 172.21.15.66:0/3514756929 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569d0df7f40 con 0x5569d0e3dd60 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd2ceff5700 10 client.15573 handle_command_reply: tid=0 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 client.15573 shutdown 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 msgr2=0x5569d0e32400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e32400 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000b960 tx=0x7fd2e000bd20 comp rx=0 tx=0).stop 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 shutdown_connections 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e3dd60 0x5569d0e40180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e39440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e31ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd2ceff5700 0 client.15573 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:16.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3514756929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e28d30 0x5569d0e32400 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 >> 172.21.15.66:0/3514756929 conn(0x5569d0d577e0 msgr2=0x5569d0e23a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:16.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 shutdown_connections 2023-12-11T19:34:16.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.788+0000 7fd304df8b80 1 -- 172.21.15.66:0/3514756929 wait complete. 2023-12-11T19:34:16.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.789+0000 7fd304df8b80 20 client.15573 trim_cache size 0 max 16384 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.809+0000 7fd304df8b80 1 Processor -- start 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.809+0000 7fd304df8b80 1 -- start start 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.809+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e28d30 0x5569d0e39400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 0x5569d0e31ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd304df8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0e1fb50 con 0x5569d0e29c20 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd304df8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0e45370 con 0x5569d0e28d30 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd304df8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0e32d60 con 0x5569d0e283d0 2023-12-11T19:34:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 0x5569d0e31ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 0x5569d0e31ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48432/0 (socket says 172.21.15.66:48432) 2023-12-11T19:34:16.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eb7fe700 1 -- 172.21.15.66:0/3617619536 learned_addr learned my addr 172.21.15.66:0/3617619536 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e28d30 0x5569d0e39400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eb7fe700 1 -- 172.21.15.66:0/3617619536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 msgr2=0x5569d0e313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.810+0000 7fd2eb7fe700 1 -- 172.21.15.66:0/3617619536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e28d30 msgr2=0x5569d0e39400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e28d30 0x5569d0e39400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2eb7fe700 1 -- 172.21.15.66:0/3617619536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d0ba5de0 con 0x5569d0e29c20 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 0x5569d0e31ec0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40094b0 tx=0x7fd2e401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3a734c91e92da6 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2d1ffb700 1 -- 172.21.15.66:0/3617619536 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4027650 con 0x5569d0e29c20 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2d1ffb700 1 -- 172.21.15.66:0/3617619536 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2e4027c40 con 0x5569d0e29c20 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.811+0000 7fd2d1ffb700 1 -- 172.21.15.66:0/3617619536 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e401abe0 con 0x5569d0e29c20 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 -- 172.21.15.66:0/3617619536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 msgr2=0x5569d0e31ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 0x5569d0e31ec0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40094b0 tx=0x7fd2e401d000 comp rx=0 tx=0).stop 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 -- 172.21.15.66:0/3617619536 shutdown_connections 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e29c20 0x5569d0e31ec0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e283d0 0x5569d0e313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 --2- 172.21.15.66:0/3617619536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e28d30 0x5569d0e39400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 -- 172.21.15.66:0/3617619536 >> 172.21.15.66:0/3617619536 conn(0x5569d0d577e0 msgr2=0x5569d0e301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 -- 172.21.15.66:0/3617619536 shutdown_connections 2023-12-11T19:34:16.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.812+0000 7fd304df8b80 1 -- 172.21.15.66:0/3617619536 wait complete. 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.813+0000 7fd304df8b80 1 -- start start 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.813+0000 7fd304df8b80 1 Processor -- start 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.813+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 0x5569d0e38d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.813+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e28d30 0x5569d0e392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd304df8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd304df8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0e37a90 con 0x5569d0e283d0 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd304df8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0e37c00 con 0x5569d0e29c20 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd304df8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d0e37dc0 con 0x5569d0e28d30 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd304df8b80 10 client.0 _ll_register_callbacks cb 0x5569d0c61a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 0x5569d0e38d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 0x5569d0e38d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48438/0 (socket says 172.21.15.66:48438) 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 -- 172.21.15.66:0/1298608591 learned_addr learned my addr 172.21.15.66:0/1298608591 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e28d30 0x5569d0e392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 -- 172.21.15.66:0/1298608591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e28d30 msgr2=0x5569d0e392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e28d30 0x5569d0e392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 -- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 msgr2=0x5569d0e373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eaffd700 1 -- 172.21.15.66:0/1298608591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e40092a0 con 0x5569d0e283d0 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e28d30 0x5569d0e392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.814+0000 7fd2eb7fe700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:16.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2eaffd700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 0x5569d0e38d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc097940 tx=0x7fd2dc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b9ea656ab009c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2ceff5700 1 -- 172.21.15.66:0/1298608591 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0a3740 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2ceff5700 1 -- 172.21.15.66:0/1298608591 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2dc0b0020 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2dc09b440 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2ceff5700 1 -- 172.21.15.66:0/1298608591 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2dc09f590 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd304df8b80 10 client.15579 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5569d0b8ec60 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2ceff5700 1 -- 172.21.15.66:0/1298608591 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2dc081020 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.815+0000 7fd2ceff5700 1 -- 172.21.15.66:0/1298608591 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2dc063070 con 0x5569d0e283d0 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd304df8b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd304df8b80 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd304df8b80 20 client.15579 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd304df8b80 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e952d0 0x5569d0e976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:16.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd304df8b80 4 client.15579 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:34:16.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569d0d42fd0 con 0x5569d0e952d0 2023-12-11T19:34:16.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.816+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e952d0 0x5569d0e976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd2ceff5700 10 client.15579 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd2ea7fc700 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e952d0 0x5569d0e976f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf94e76e137b4326 server_cookie=fdb8b78edd58de5 in_seq=0 out_seq=0 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd2ceff5700 1 -- 172.21.15.66:0/1298608591 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5569d0d42fd0 con 0x5569d0e952d0 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd2ceff5700 10 client.15579 handle_command_reply: tid=0 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 client.15579 shutdown 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 msgr2=0x5569d0e38d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 0x5569d0e38d90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc097940 tx=0x7fd2dc005e80 comp rx=0 tx=0).stop 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 shutdown_connections 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5569d0e952d0 0x5569d0e976f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5569d0e29c20 0x5569d0e373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5569d0e283d0 0x5569d0e38d90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 --2- 172.21.15.66:0/1298608591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569d0e28d30 0x5569d0e392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 >> 172.21.15.66:0/1298608591 conn(0x5569d0d577e0 msgr2=0x5569d0e2cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd2ceff5700 0 client.15579 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 shutdown_connections 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.817+0000 7fd304df8b80 1 -- 172.21.15.66:0/1298608591 wait complete. 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd304df8b80 20 client.15579 trim_cache size 0 max 16384 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-12-11T19:34:16.821 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d40a1480 msgr2=0x7fd2d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d40a1480 0x7fd2d40a3940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009fd0 tx=0x7fd2e4009c60 comp rx=0 tx=0).stop 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 msgr2=0x7fd2ec075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec075330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009f50 tx=0x7fd2dc005b00 comp rx=0 tx=0).stop 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 shutdown_connections 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd2d40a1480 0x7fd2d40a3940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ec103130 0x7fd2ec073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd2ec1015a0 0x7fd2ec075330 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 --2- 172.21.15.66:0/627478421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ec102790 0x7fd2ec0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:16.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.818+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 >> 172.21.15.66:0/627478421 conn(0x7fd2ec0fcb10 msgr2=0x7fd2ec105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:16.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.819+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 shutdown_connections 2023-12-11T19:34:16.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:16.819+0000 7fd2f13ff700 1 -- 172.21.15.66:0/627478421 wait complete. 2023-12-11T19:34:16.837 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-11T19:34:16.838 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.057+0000 7fd6049fe700 1 -- 172.21.15.66:0/2373402483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 msgr2=0x7fd6001095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.057+0000 7fd6049fe700 1 --2- 172.21.15.66:0/2373402483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001095e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800b0a0 tx=0x7fd5f800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 -- 172.21.15.66:0/2373402483 shutdown_connections 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 --2- 172.21.15.66:0/2373402483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001095e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 --2- 172.21.15.66:0/2373402483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd600101f90 0x7fd600104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 --2- 172.21.15.66:0/2373402483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6001048c0 0x7fd600106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 -- 172.21.15.66:0/2373402483 >> 172.21.15.66:0/2373402483 conn(0x7fd6000fb8e0 msgr2=0x7fd6000fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 -- 172.21.15.66:0/2373402483 shutdown_connections 2023-12-11T19:34:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.058+0000 7fd6049fe700 1 -- 172.21.15.66:0/2373402483 wait complete. 2023-12-11T19:34:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 Processor -- start 2023-12-11T19:34:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 -- start start 2023-12-11T19:34:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd600101f90 0x7fd6001995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6001048c0 0x7fd600199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001a1110 con 0x7fd6001071f0 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001a1250 con 0x7fd600101f90 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.059+0000 7fd6049fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001a13e0 con 0x7fd6001048c0 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5ff59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd600101f90 0x7fd6001995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5ff59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48442/0 (socket says 172.21.15.66:48442) 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5ff59e700 1 -- 172.21.15.66:0/1237959080 learned_addr learned my addr 172.21.15.66:0/1237959080 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fe59c700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6001048c0 0x7fd600199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 -- 172.21.15.66:0/1237959080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6001048c0 msgr2=0x7fd600199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6001048c0 0x7fd600199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 -- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 msgr2=0x7fd6001a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 -- 172.21.15.66:0/1237959080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8009d00 con 0x7fd600101f90 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5ff59e700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:17.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.060+0000 7fd5fed9d700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd600101f90 0x7fd6001995e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000e9f0 tx=0x7fd5f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.061+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000c9c0 con 0x7fd600101f90 2023-12-11T19:34:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.061+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6001a16c0 con 0x7fd600101f90 2023-12-11T19:34:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.061+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5f0010430 con 0x7fd600101f90 2023-12-11T19:34:17.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.061+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0018680 con 0x7fd600101f90 2023-12-11T19:34:17.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.061+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6001a1bd0 con 0x7fd600101f90 2023-12-11T19:34:17.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.062+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd600108d70 con 0x7fd600101f90 2023-12-11T19:34:17.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.064+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd5f00105a0 con 0x7fd600101f90 2023-12-11T19:34:17.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.065+0000 7fd5e7fff700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd5e80a1480 0x7fd5e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.066+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd5f0014070 con 0x7fd600101f90 2023-12-11T19:34:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.066+0000 7fd5fe59c700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd5e80a1480 0x7fd5e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.067+0000 7fd5fe59c700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd5e80a1480 0x7fd5e80a3940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4007be0 tx=0x7fd5f40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.068+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd5f0096780 con 0x7fd600101f90 2023-12-11T19:34:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.392+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd60010cb80 con 0x7fd600101f90 2023-12-11T19:34:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.393+0000 7fd5e7fff700 1 -- 172.21.15.66:0/1237959080 <== mon.2 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7fd5f0099da0 con 0x7fd600101f90 2023-12-11T19:34:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:34:17.394 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd5e80a1480 msgr2=0x7fd5e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd5e80a1480 0x7fd5e80a3940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4007be0 tx=0x7fd5f40073d0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd600101f90 msgr2=0x7fd6001995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd600101f90 0x7fd6001995e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000e9f0 tx=0x7fd5f000ed00 comp rx=0 tx=0).stop 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 shutdown_connections 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd5e80a1480 0x7fd5e80a3940 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd600101f90 0x7fd6001995e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd6001071f0 0x7fd6001a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 --2- 172.21.15.66:0/1237959080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6001048c0 0x7fd600199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 >> 172.21.15.66:0/1237959080 conn(0x7fd6000fb8e0 msgr2=0x7fd600101720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 shutdown_connections 2023-12-11T19:34:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.396+0000 7fd6049fe700 1 -- 172.21.15.66:0/1237959080 wait complete. 2023-12-11T19:34:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:34:17.416 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-11T19:34:17.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 -- 172.21.15.66:0/2389972071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250075610 msgr2=0x7fb250073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 --2- 172.21.15.66:0/2389972071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250075610 0x7fb250073a60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb240009990 tx=0x7fb240009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 -- 172.21.15.66:0/2389972071 shutdown_connections 2023-12-11T19:34:17.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 --2- 172.21.15.66:0/2389972071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb250109460 0x7fb25010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 --2- 172.21.15.66:0/2389972071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250075610 0x7fb250073a60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 --2- 172.21.15.66:0/2389972071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb2500744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 -- 172.21.15.66:0/2389972071 >> 172.21.15.66:0/2389972071 conn(0x7fb2500fcb10 msgr2=0x7fb2500fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 -- 172.21.15.66:0/2389972071 shutdown_connections 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.633+0000 7fb2563ff700 1 -- 172.21.15.66:0/2389972071 wait complete. 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.634+0000 7fb2563ff700 1 Processor -- start 2023-12-11T19:34:17.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 -- start start 2023-12-11T19:34:17.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb250199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb250075610 0x7fb250199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250109460 0x7fb2501a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501a1170 con 0x7fb250109460 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501a12b0 con 0x7fb250074030 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb2563ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501a1440 con 0x7fb250075610 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb24ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb250199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb24ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb250199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:51616/0 (socket says 172.21.15.66:51616) 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.635+0000 7fb24ffff700 1 -- 172.21.15.66:0/405944440 learned_addr learned my addr 172.21.15.66:0/405944440 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb250075610 0x7fb250199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb25499c700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250109460 0x7fb2501a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24ffff700 1 -- 172.21.15.66:0/405944440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb250075610 msgr2=0x7fb250199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24ffff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb250075610 0x7fb250199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24ffff700 1 -- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250109460 msgr2=0x7fb2501a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24ffff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250109460 0x7fb2501a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24ffff700 1 -- 172.21.15.66:0/405944440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb244009520 con 0x7fb250074030 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.636+0000 7fb24ffff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb250199640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb240006010 tx=0x7fb240004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.637+0000 7fb24d7fa700 1 -- 172.21.15.66:0/405944440 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24001d070 con 0x7fb250074030 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.637+0000 7fb24d7fa700 1 -- 172.21.15.66:0/405944440 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb24000bdb0 con 0x7fb250074030 2023-12-11T19:34:17.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.637+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2400095f0 con 0x7fb250074030 2023-12-11T19:34:17.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.637+0000 7fb24d7fa700 1 -- 172.21.15.66:0/405944440 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000fda0 con 0x7fb250074030 2023-12-11T19:34:17.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.637+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2501a19b0 con 0x7fb250074030 2023-12-11T19:34:17.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.642+0000 7fb24d7fa700 1 -- 172.21.15.66:0/405944440 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb24000f460 con 0x7fb250074030 2023-12-11T19:34:17.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 Processor -- start 2023-12-11T19:34:17.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 -- start start 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 0x55c06e7a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a1d30 0x55c06e7a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6fcb50 con 0x55c06e7a1d30 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6fa470 con 0x55c06e7a2c20 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.643+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6f5d60 con 0x55c06e7a13d0 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 0x55c06e7a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 0x55c06e7a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:49572/0 (socket says 172.21.15.66:49572) 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 -- 172.21.15.66:0/2617613181 learned_addr learned my addr 172.21.15.66:0/2617613181 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb24ffff700 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a1d30 0x55c06e7a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 -- 172.21.15.66:0/2617613181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 msgr2=0x55c06e7a30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 -- 172.21.15.66:0/2617613181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a1d30 msgr2=0x55c06e7a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a1d30 0x55c06e7a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 -- 172.21.15.66:0/2617613181 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06e572630 con 0x55c06e7a13d0 2023-12-11T19:34:17.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.644+0000 7fb25499c700 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 0x55c06e7a17f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb248011680 tx=0x7fb248009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce9e79ca52253877 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.645+0000 7fb24d7fa700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb2380a1500 0x7fb2380a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.645+0000 7fb2367fc700 1 -- 172.21.15.66:0/2617613181 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800e040 con 0x55c06e7a13d0 2023-12-11T19:34:17.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.645+0000 7fb2367fc700 1 -- 172.21.15.66:0/2617613181 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb24800f040 con 0x55c06e7a13d0 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.645+0000 7fb2367fc700 1 -- 172.21.15.66:0/2617613181 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248004770 con 0x55c06e7a13d0 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.645+0000 7fb24d7fa700 1 -- 172.21.15.66:0/405944440 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb2400cdfa0 con 0x7fb250074030 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.645+0000 7fb24ffff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb2380a1500 0x7fb2380a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2617613181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 msgr2=0x55c06e7a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 0x55c06e7a17f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb248011680 tx=0x7fb248009920 comp rx=0 tx=0).stop 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb24ffff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb2380a1500 0x7fb2380a39c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b2e0 tx=0x7fb240005910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2617613181 shutdown_connections 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a13d0 0x55c06e7a17f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2617613181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a1d30 0x55c06e7a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2617613181 >> 172.21.15.66:0/2617613181 conn(0x55c06e6d07e0 msgr2=0x55c06e79dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2617613181 shutdown_connections 2023-12-11T19:34:17.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.646+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2617613181 wait complete. 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.647+0000 7fb269d1bb80 1 -- start start 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.647+0000 7fb269d1bb80 1 Processor -- start 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a1d30 0x55c06e7ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a2c20 0x55c06e7b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6af560 con 0x55c06e7a13d0 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e6af200 con 0x55c06e7a1d30 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e574920 con 0x55c06e7a2c20 2023-12-11T19:34:17.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb269d1bb80 10 client.0 _ll_register_callbacks cb 0x55c06e5daa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb25499c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb25499c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48466/0 (socket says 172.21.15.66:48466) 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb25499c700 1 -- 172.21.15.66:0/1550984797 learned_addr learned my addr 172.21.15.66:0/1550984797 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb25499c700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a2c20 0x55c06e7b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.648+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a1d30 0x55c06e7ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb25499c700 1 -- 172.21.15.66:0/1550984797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a2c20 msgr2=0x55c06e7b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb25499c700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a2c20 0x55c06e7b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb25499c700 1 -- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a1d30 msgr2=0x55c06e7ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb25499c700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a1d30 0x55c06e7ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb25499c700 1 -- 172.21.15.66:0/1550984797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb248011330 con 0x55c06e7a13d0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb25499c700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ab180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb248006080 tx=0x7fb248009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2880cc079dee1e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb22f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb22f7fe700 1 -- 172.21.15.66:0/1550984797 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800e040 con 0x55c06e7a13d0 2023-12-11T19:34:17.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c06e794b40 con 0x55c06e7a13d0 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.649+0000 7fb22f7fe700 1 -- 172.21.15.66:0/1550984797 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb248007500 con 0x55c06e7a13d0 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb22f7fe700 1 -- 172.21.15.66:0/1550984797 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb248003f20 con 0x55c06e7a13d0 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 10 client.15597 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c06e51ede0 con 0x55c06e7a13d0 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb22f7fe700 1 -- 172.21.15.66:0/1550984797 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c06e794b40 con 0x55c06e7a13d0 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb22f7fe700 1 -- 172.21.15.66:0/1550984797 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb2480155e0 con 0x55c06e7a13d0 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 10 client.15597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 20 client.15597 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e7b6e50 0x55c06e7b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 4 client.15597 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.650+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c06e770f40 con 0x55c06e7b6e50 2023-12-11T19:34:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.651+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e7b6e50 0x55c06e7b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.651+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e7b6e50 0x55c06e7b9270 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19833acc0705beca server_cookie=7045deb035282a48 in_seq=0 out_seq=0 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.651+0000 7fb22f7fe700 10 client.15597 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.652+0000 7fb22f7fe700 1 -- 172.21.15.66:0/1550984797 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c06e770f40 con 0x55c06e7b6e50 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.652+0000 7fb22f7fe700 10 client.15597 handle_command_reply: tid=0 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.652+0000 7fb269d1bb80 1 client.15597 shutdown 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 msgr2=0x55c06e7ab180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ab180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb248006080 tx=0x7fb248009dd0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 shutdown_connections 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e7b6e50 0x55c06e7b9270 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a2c20 0x55c06e7b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ab180 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/1550984797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a1d30 0x55c06e7ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb22f7fe700 0 client.15597 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:17.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 >> 172.21.15.66:0/1550984797 conn(0x55c06e6d07e0 msgr2=0x55c06e79ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 shutdown_connections 2023-12-11T19:34:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 1 -- 172.21.15.66:0/1550984797 wait complete. 2023-12-11T19:34:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.653+0000 7fb269d1bb80 20 client.15597 trim_cache size 0 max 16384 2023-12-11T19:34:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.680+0000 7fb269d1bb80 1 Processor -- start 2023-12-11T19:34:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.680+0000 7fb269d1bb80 1 -- start start 2023-12-11T19:34:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.680+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.680+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.680+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.680+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e7aacf0 con 0x55c06e7a13d0 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e798b50 con 0x55c06e7a2c20 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e7a0f70 con 0x55c06e7a1d30 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48474/0 (socket says 172.21.15.66:48474) 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 -- 172.21.15.66:0/511727564 learned_addr learned my addr 172.21.15.66:0/511727564 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 -- 172.21.15.66:0/511727564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 msgr2=0x55c06e7b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 -- 172.21.15.66:0/511727564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 msgr2=0x55c06e7a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 -- 172.21.15.66:0/511727564 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06e507c60 con 0x55c06e7a13d0 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.681+0000 7fb25499c700 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7b2db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb2480061e0 tx=0x7fb2480095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaff554f1678d26d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.682+0000 7fb236ffd700 1 -- 172.21.15.66:0/511727564 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800e040 con 0x55c06e7a13d0 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.682+0000 7fb236ffd700 1 -- 172.21.15.66:0/511727564 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb248027070 con 0x55c06e7a13d0 2023-12-11T19:34:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.682+0000 7fb236ffd700 1 -- 172.21.15.66:0/511727564 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480041f0 con 0x55c06e7a13d0 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.682+0000 7fb269d1bb80 1 -- 172.21.15.66:0/511727564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 msgr2=0x55c06e7b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.682+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7b2db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb2480061e0 tx=0x7fb2480095e0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 -- 172.21.15.66:0/511727564 shutdown_connections 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7b2db0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/511727564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 -- 172.21.15.66:0/511727564 >> 172.21.15.66:0/511727564 conn(0x55c06e6d07e0 msgr2=0x55c06e7bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 -- 172.21.15.66:0/511727564 shutdown_connections 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 -- 172.21.15.66:0/511727564 wait complete. 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 -- start start 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.683+0000 7fb269d1bb80 1 Processor -- start 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e7b1be0 con 0x55c06e7a13d0 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e7b1d50 con 0x55c06e7a2c20 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06e7b1f10 con 0x55c06e7a1d30 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb269d1bb80 10 client.0 _ll_register_callbacks cb 0x55c06e5daa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb25499c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb25499c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48488/0 (socket says 172.21.15.66:48488) 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb25499c700 1 -- 172.21.15.66:0/2778467989 learned_addr learned my addr 172.21.15.66:0/2778467989 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.684+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb25499c700 1 -- 172.21.15.66:0/2778467989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 msgr2=0x55c06e7ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb25499c700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb25499c700 1 -- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 msgr2=0x55c06e7b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb25499c700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb25499c700 1 -- 172.21.15.66:0/2778467989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c06e51ede0 con 0x55c06e7a13d0 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb25499c700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ae190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb248009c10 tx=0x7fb248015e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8659bc9cc372b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb22f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb22f7fe700 1 -- 172.21.15.66:0/2778467989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248004400 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb24800f040 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb22f7fe700 1 -- 172.21.15.66:0/2778467989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb248015450 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb22f7fe700 1 -- 172.21.15.66:0/2778467989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb248015610 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb269d1bb80 10 client.15603 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.685+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c06e545250 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb22f7fe700 1 -- 172.21.15.66:0/2778467989 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb24800f040 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb22f7fe700 1 -- 172.21.15.66:0/2778467989 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb24802c410 con 0x55c06e7a13d0 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb269d1bb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb269d1bb80 10 client.15603 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb269d1bb80 20 client.15603 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e80dab0 0x55c06e80fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb269d1bb80 4 client.15603 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c06e6bbfd0 con 0x55c06e80dab0 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.686+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e80dab0 0x55c06e80fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb24f7fe700 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e80dab0 0x55c06e80fed0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d734c57cb6537a5f server_cookie=9348b5d0c810e409 in_seq=0 out_seq=0 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb22f7fe700 10 client.15603 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb22f7fe700 1 -- 172.21.15.66:0/2778467989 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c06e6bbfd0 con 0x55c06e80dab0 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb22f7fe700 10 client.15603 handle_command_reply: tid=0 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb269d1bb80 1 client.15603 shutdown 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 msgr2=0x55c06e7ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.687+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ae190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb248009c10 tx=0x7fb248015e00 comp rx=0 tx=0).stop 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 shutdown_connections 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55c06e80dab0 0x55c06e80fed0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c06e7a2c20 0x55c06e7b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c06e7a13d0 0x55c06e7ae190 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb22f7fe700 0 client.15603 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 --2- 172.21.15.66:0/2778467989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06e7a1d30 0x55c06e7ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 >> 172.21.15.66:0/2778467989 conn(0x55c06e6d07e0 msgr2=0x55c06e7a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 shutdown_connections 2023-12-11T19:34:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 1 -- 172.21.15.66:0/2778467989 wait complete. 2023-12-11T19:34:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.688+0000 7fb269d1bb80 20 client.15603 trim_cache size 0 max 16384 2023-12-11T19:34:17.695 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb2380a1500 msgr2=0x7fb2380a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb2380a1500 0x7fb2380a39c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb24000b2e0 tx=0x7fb240005910 comp rx=0 tx=0).stop 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 msgr2=0x7fb250199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb250199640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb240006010 tx=0x7fb240004730 comp rx=0 tx=0).stop 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 shutdown_connections 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb2380a1500 0x7fb2380a39c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb250074030 0x7fb250199640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250109460 0x7fb2501a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 --2- 172.21.15.66:0/405944440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb250075610 0x7fb250199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 >> 172.21.15.66:0/405944440 conn(0x7fb2500fcb10 msgr2=0x7fb250105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 shutdown_connections 2023-12-11T19:34:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.689+0000 7fb2563ff700 1 -- 172.21.15.66:0/405944440 wait complete. 2023-12-11T19:34:17.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:17 smithi066 ceph-mon[97920]: pgmap v381: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:17.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:17 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1237959080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:34:17.707 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.931+0000 7f9cb59ff700 1 -- 172.21.15.66:0/1189152058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0102010 msgr2=0x7f9cb0104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.931+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/1189152058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0102010 0x7f9cb0104400 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009990 tx=0x7f9ca0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 -- 172.21.15.66:0/1189152058 shutdown_connections 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/1189152058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0107270 0x7f9cb01096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/1189152058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0102010 0x7f9cb0104400 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/1189152058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0104940 0x7f9cb0106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 -- 172.21.15.66:0/1189152058 >> 172.21.15.66:0/1189152058 conn(0x7f9cb00fba20 msgr2=0x7f9cb00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 -- 172.21.15.66:0/1189152058 shutdown_connections 2023-12-11T19:34:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.932+0000 7f9cb59ff700 1 -- 172.21.15.66:0/1189152058 wait complete. 2023-12-11T19:34:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.933+0000 7f9cb59ff700 1 Processor -- start 2023-12-11T19:34:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 -- start start 2023-12-11T19:34:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 0x7f9cb0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0104940 0x7f9cb0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0107270 0x7f9cb019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb019ef60 con 0x7f9cb0107270 2023-12-11T19:34:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb019f0a0 con 0x7f9cb0104940 2023-12-11T19:34:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.934+0000 7f9cb59ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb019f230 con 0x7f9cb0102010 2023-12-11T19:34:17.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 0x7f9cb0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0107270 0x7f9cb019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 0x7f9cb0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:49624/0 (socket says 172.21.15.66:49624) 2023-12-11T19:34:17.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 -- 172.21.15.66:0/361575086 learned_addr learned my addr 172.21.15.66:0/361575086 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0104940 0x7f9cb0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 -- 172.21.15.66:0/361575086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0104940 msgr2=0x7f9cb0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0104940 0x7f9cb0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 -- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0107270 msgr2=0x7f9cb019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0107270 0x7f9cb019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 -- 172.21.15.66:0/361575086 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca00095f0 con 0x7f9cb0102010 2023-12-11T19:34:17.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0104940 0x7f9cb0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:17.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.935+0000 7f9caeffd700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 0x7f9cb0197430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0003ec0 tx=0x7f9ca0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.936+0000 7f9cb49fd700 1 -- 172.21.15.66:0/361575086 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca001d070 con 0x7f9cb0102010 2023-12-11T19:34:17.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.936+0000 7f9cb49fd700 1 -- 172.21.15.66:0/361575086 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca000bdb0 con 0x7f9cb0102010 2023-12-11T19:34:17.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.936+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb019f4b0 con 0x7f9cb0102010 2023-12-11T19:34:17.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.936+0000 7f9cb49fd700 1 -- 172.21.15.66:0/361575086 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0021410 con 0x7f9cb0102010 2023-12-11T19:34:17.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.936+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb019f8d0 con 0x7f9cb0102010 2023-12-11T19:34:17.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.939+0000 7f9cb49fd700 1 -- 172.21.15.66:0/361575086 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9ca000f660 con 0x7f9cb0102010 2023-12-11T19:34:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.941+0000 7f9cc93f6b80 1 Processor -- start 2023-12-11T19:34:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.941+0000 7f9cb49fd700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9c980a1480 0x7f9c980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.941+0000 7f9cc93f6b80 1 -- start start 2023-12-11T19:34:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.941+0000 7f9cb49fd700 1 -- 172.21.15.66:0/361575086 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ca00ce060 con 0x7f9cb0102010 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.941+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9c980a1480 0x7f9c980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 0x55564a8487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a849640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a849c20 0x55564a84a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a7a3b50 con 0x55564a8483d0 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a7a1470 con 0x55564a849c20 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a79cd60 con 0x55564a848d30 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 0x55564a8487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 0x55564a8487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48518/0 (socket says 172.21.15.66:48518) 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caeffd700 1 -- 172.21.15.66:0/3363403027 learned_addr learned my addr 172.21.15.66:0/3363403027 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a849c20 0x55564a84a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9c980a1480 0x7f9c980a3940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800f590 tx=0x7f9ca800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a849640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caeffd700 1 -- 172.21.15.66:0/3363403027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 msgr2=0x55564a849640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.942+0000 7f9caeffd700 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a849640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9caeffd700 1 -- 172.21.15.66:0/3363403027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a849c20 msgr2=0x55564a84a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9caeffd700 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a849c20 0x55564a84a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9caeffd700 1 -- 172.21.15.66:0/3363403027 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55564a619630 con 0x55564a8483d0 2023-12-11T19:34:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9caeffd700 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 0x55564a8487f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00978d0 tx=0x7f9ca009af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fa435cb4d2764b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9c95ffb700 1 -- 172.21.15.66:0/3363403027 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000f460 con 0x55564a8483d0 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9c95ffb700 1 -- 172.21.15.66:0/3363403027 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca009bad0 con 0x55564a8483d0 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.943+0000 7f9c95ffb700 1 -- 172.21.15.66:0/3363403027 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0099800 con 0x55564a8483d0 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3363403027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 msgr2=0x55564a8487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 0x55564a8487f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00978d0 tx=0x7f9ca009af10 comp rx=0 tx=0).stop 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3363403027 shutdown_connections 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a849c20 0x55564a84a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a8483d0 0x55564a8487f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3363403027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a849640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3363403027 >> 172.21.15.66:0/3363403027 conn(0x55564a7777e0 msgr2=0x55564a844d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3363403027 shutdown_connections 2023-12-11T19:34:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.944+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3363403027 wait complete. 2023-12-11T19:34:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.945+0000 7f9cc93f6b80 1 -- start start 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.945+0000 7f9cc93f6b80 1 Processor -- start 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a851eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 0x55564a8523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a849c20 0x55564a859430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a756560 con 0x55564a848d30 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a756200 con 0x55564a8483d0 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a61b920 con 0x55564a849c20 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cc93f6b80 10 client.0 _ll_register_callbacks cb 0x55564a681a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.946+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 0x55564a8523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a851eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 0x55564a8523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48530/0 (socket says 172.21.15.66:48530) 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9cae7fc700 1 -- 172.21.15.66:0/2860516559 learned_addr learned my addr 172.21.15.66:0/2860516559 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a849c20 0x55564a859430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 -- 172.21.15.66:0/2860516559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a849c20 msgr2=0x55564a859430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a849c20 0x55564a859430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 -- 172.21.15.66:0/2860516559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 msgr2=0x55564a8523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 0x55564a8523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 -- 172.21.15.66:0/2860516559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb019f8d0 con 0x55564a8483d0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 0x55564a8523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9caeffd700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a851eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0097e40 tx=0x7f9ca000f4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d436a4e3ce2fc013 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.947+0000 7f9c8effd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9c8effd700 1 -- 172.21.15.66:0/2860516559 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca009a8c0 con 0x55564a8483d0 2023-12-11T19:34:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9c8effd700 1 -- 172.21.15.66:0/2860516559 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ca009aa20 con 0x55564a8483d0 2023-12-11T19:34:17.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55564a83bb40 con 0x55564a8483d0 2023-12-11T19:34:17.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9c8effd700 1 -- 172.21.15.66:0/2860516559 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca00a3240 con 0x55564a8483d0 2023-12-11T19:34:17.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9cc93f6b80 10 client.25373 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:17.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55564a5c5de0 con 0x55564a8483d0 2023-12-11T19:34:17.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9c8effd700 1 -- 172.21.15.66:0/2860516559 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca0081020 con 0x55564a8483d0 2023-12-11T19:34:17.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.948+0000 7f9c8effd700 1 -- 172.21.15.66:0/2860516559 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ca00b3050 con 0x55564a8483d0 2023-12-11T19:34:17.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cc93f6b80 10 client.25373 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cc93f6b80 10 client.25373 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cc93f6b80 20 client.25373 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a85d850 0x55564a85fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cc93f6b80 4 client.25373 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55564a817f40 con 0x55564a85d850 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.949+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a85d850 0x55564a85fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.950+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a85d850 0x55564a85fc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77a19cc347f11df8 server_cookie=c9a8d19cae130c14 in_seq=0 out_seq=0 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.950+0000 7f9c8effd700 10 client.25373 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.950+0000 7f9c8effd700 1 -- 172.21.15.66:0/2860516559 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55564a817f40 con 0x55564a85d850 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.950+0000 7f9c8effd700 10 client.25373 handle_command_reply: tid=0 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.950+0000 7f9cc93f6b80 1 client.25373 shutdown 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 msgr2=0x55564a851eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a851eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0097e40 tx=0x7f9ca000f4f0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 shutdown_connections 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a85d850 0x55564a85fc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a851eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9c8effd700 0 client.25373 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a849c20 0x55564a859430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/2860516559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a848d30 0x55564a8523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 >> 172.21.15.66:0/2860516559 conn(0x55564a7777e0 msgr2=0x55564a843a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 shutdown_connections 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/2860516559 wait complete. 2023-12-11T19:34:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.951+0000 7f9cc93f6b80 20 client.25373 trim_cache size 0 max 16384 2023-12-11T19:34:17.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 Processor -- start 2023-12-11T19:34:17.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 -- start start 2023-12-11T19:34:17.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a8483d0 0x55564a859ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a848d30 0x55564a8593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a84a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a83fb50 con 0x55564a849c20 2023-12-11T19:34:17.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a84fe20 con 0x55564a848d30 2023-12-11T19:34:17.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.979+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a84d730 con 0x55564a8483d0 2023-12-11T19:34:17.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a84a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a84a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48546/0 (socket says 172.21.15.66:48546) 2023-12-11T19:34:17.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 -- 172.21.15.66:0/3972318092 learned_addr learned my addr 172.21.15.66:0/3972318092 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a848d30 0x55564a8593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caeffd700 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a8483d0 0x55564a859ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 -- 172.21.15.66:0/3972318092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a8483d0 msgr2=0x55564a859ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a8483d0 0x55564a859ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 -- 172.21.15.66:0/3972318092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a848d30 msgr2=0x55564a8593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a848d30 0x55564a8593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caf7fe700 1 -- 172.21.15.66:0/3972318092 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55564a5aec60 con 0x55564a849c20 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.980+0000 7f9caeffd700 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a8483d0 0x55564a859ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.981+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a84a000 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009450 tx=0x7f9ca8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a976af2337f04422 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.981+0000 7f9c967fc700 1 -- 172.21.15.66:0/3972318092 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800e040 con 0x55564a849c20 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.981+0000 7f9c967fc700 1 -- 172.21.15.66:0/3972318092 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca800f040 con 0x55564a849c20 2023-12-11T19:34:17.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.981+0000 7f9c967fc700 1 -- 172.21.15.66:0/3972318092 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8023440 con 0x55564a849c20 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3972318092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 msgr2=0x55564a84a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a84a000 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009450 tx=0x7f9ca8014000 comp rx=0 tx=0).stop 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3972318092 shutdown_connections 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a84a000 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a8483d0 0x55564a859ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/3972318092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a848d30 0x55564a8593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3972318092 >> 172.21.15.66:0/3972318092 conn(0x55564a7777e0 msgr2=0x55564a8501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3972318092 shutdown_connections 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/3972318092 wait complete. 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.982+0000 7f9cc93f6b80 1 -- start start 2023-12-11T19:34:17.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 Processor -- start 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a8570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a857620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a858540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a858a80 con 0x55564a849c20 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a858bf0 con 0x55564a8483d0 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.983+0000 7f9cc93f6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564a858db0 con 0x55564a848d30 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9cc93f6b80 10 client.0 _ll_register_callbacks cb 0x55564a681a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a858540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a857620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a858540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48550/0 (socket says 172.21.15.66:48550) 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 -- 172.21.15.66:0/977451022 learned_addr learned my addr 172.21.15.66:0/977451022 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:17.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caeffd700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a8570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 -- 172.21.15.66:0/977451022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 msgr2=0x55564a857620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a857620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 -- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 msgr2=0x55564a8570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a8570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9caf7fe700 1 -- 172.21.15.66:0/977451022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca8009130 con 0x55564a849c20 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.984+0000 7f9cae7fc700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a857620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a858540 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009960 tx=0x7f9ca8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e515fa767a55c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9c8effd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9c8effd700 1 -- 172.21.15.66:0/977451022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800e030 con 0x55564a849c20 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ca009bad0 con 0x55564a849c20 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9c8effd700 1 -- 172.21.15.66:0/977451022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ca801c040 con 0x55564a849c20 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9c8effd700 1 -- 172.21.15.66:0/977451022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca8004a90 con 0x55564a849c20 2023-12-11T19:34:17.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9cc93f6b80 10 client.15621 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.985+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55564a5ec250 con 0x55564a849c20 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9c8effd700 1 -- 172.21.15.66:0/977451022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca009bad0 con 0x55564a849c20 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9c8effd700 1 -- 172.21.15.66:0/977451022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ca802cb90 con 0x55564a849c20 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9cc93f6b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9cc93f6b80 10 client.15621 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9cc93f6b80 20 client.15621 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a8b4ae0 0x55564a8b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9cc93f6b80 4 client.15621 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55564a762fd0 con 0x55564a8b4ae0 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.986+0000 7f9caeffd700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a8b4ae0 0x55564a8b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9caeffd700 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a8b4ae0 0x55564a8b6f00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5481414d11c8a489 server_cookie=1fc4b0cb55539f90 in_seq=0 out_seq=0 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9c8effd700 10 client.15621 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:17.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9c8effd700 1 -- 172.21.15.66:0/977451022 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55564a762fd0 con 0x55564a8b4ae0 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9c8effd700 10 client.15621 handle_command_reply: tid=0 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 client.15621 shutdown 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 msgr2=0x55564a858540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a858540 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009960 tx=0x7f9ca8003f60 comp rx=0 tx=0).stop 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 shutdown_connections 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55564a8b4ae0 0x55564a8b6f00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55564a8483d0 0x55564a8570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55564a849c20 0x55564a858540 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 --2- 172.21.15.66:0/977451022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55564a848d30 0x55564a857620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 >> 172.21.15.66:0/977451022 conn(0x55564a7777e0 msgr2=0x55564a850030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.987+0000 7f9c8effd700 0 client.15621 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 shutdown_connections 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cc93f6b80 1 -- 172.21.15.66:0/977451022 wait complete. 2023-12-11T19:34:17.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cc93f6b80 20 client.15621 trim_cache size 0 max 16384 2023-12-11T19:34:17.992 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:17.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9c980a1480 msgr2=0x7f9c980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9c980a1480 0x7f9c980a3940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800f590 tx=0x7f9ca800b040 comp rx=0 tx=0).stop 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 msgr2=0x7f9cb0197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.988+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 0x7f9cb0197430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0003ec0 tx=0x7f9ca0005740 comp rx=0 tx=0).stop 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 shutdown_connections 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9c980a1480 0x7f9c980a3940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb0107270 0x7f9cb019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb0102010 0x7f9cb0197430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 --2- 172.21.15.66:0/361575086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb0104940 0x7f9cb0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 >> 172.21.15.66:0/361575086 conn(0x7f9cb00fba20 msgr2=0x7f9cb0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 shutdown_connections 2023-12-11T19:34:17.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:17.989+0000 7f9cb59ff700 1 -- 172.21.15.66:0/361575086 wait complete. 2023-12-11T19:34:18.008 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-11T19:34:18.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:17 smithi167 ceph-mon[102750]: pgmap v381: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:18.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:17 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1237959080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:34:18.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:17 smithi070 ceph-mon[104044]: pgmap v381: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:18.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:17 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1237959080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/1380226113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54102000 msgr2=0x7fdb541043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/1380226113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54102000 0x7fdb541043f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009990 tx=0x7fdb44009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/1380226113 shutdown_connections 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/1380226113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54107260 0x7fdb541096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/1380226113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54102000 0x7fdb541043f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/1380226113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54104930 0x7fdb54106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/1380226113 >> 172.21.15.66:0/1380226113 conn(0x7fdb540fb9f0 msgr2=0x7fdb540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.230+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/1380226113 shutdown_connections 2023-12-11T19:34:18.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.231+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/1380226113 wait complete. 2023-12-11T19:34:18.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.232+0000 7fdb5a3ff700 1 Processor -- start 2023-12-11T19:34:18.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.232+0000 7fdb5a3ff700 1 -- start start 2023-12-11T19:34:18.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5a3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54102000 0x7fdb54101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5a3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54104930 0x7fdb540fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5a3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 0x7fdb541004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5a3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb54100c90 con 0x7fdb54107260 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5a3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb54100e00 con 0x7fdb54102000 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5a3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a1470 con 0x7fdb54104930 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5899c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 0x7fdb541004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5899c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 0x7fdb541004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48564/0 (socket says 172.21.15.66:48564) 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb5899c700 1 -- 172.21.15.66:0/3277330811 learned_addr learned my addr 172.21.15.66:0/3277330811 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb53fff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54102000 0x7fdb54101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.233+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54104930 0x7fdb540fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5899c700 1 -- 172.21.15.66:0/3277330811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54104930 msgr2=0x7fdb540fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5899c700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54104930 0x7fdb540fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5899c700 1 -- 172.21.15.66:0/3277330811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54102000 msgr2=0x7fdb54101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5899c700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54102000 0x7fdb54101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5899c700 1 -- 172.21.15.66:0/3277330811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440095f0 con 0x7fdb54107260 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54104930 0x7fdb540fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb53fff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54102000 0x7fdb54101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5899c700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 0x7fdb541004f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c007ee0 tx=0x7fdb4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb517fa700 1 -- 172.21.15.66:0/3277330811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb54107260 2023-12-11T19:34:18.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb541a1700 con 0x7fdb54107260 2023-12-11T19:34:18.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb517fa700 1 -- 172.21.15.66:0/3277330811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb4c00f040 con 0x7fdb54107260 2023-12-11T19:34:18.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb517fa700 1 -- 172.21.15.66:0/3277330811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c005570 con 0x7fdb54107260 2023-12-11T19:34:18.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.234+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb541a1c10 con 0x7fdb54107260 2023-12-11T19:34:18.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.238+0000 7fdb517fa700 1 -- 172.21.15.66:0/3277330811 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fdb4c00db70 con 0x7fdb54107260 2023-12-11T19:34:18.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.240+0000 7fdb6dcf9b80 1 Processor -- start 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.240+0000 7fdb517fa700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.240+0000 7fdb6dcf9b80 1 -- start start 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.240+0000 7fdb517fa700 1 -- 172.21.15.66:0/3277330811 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb4c0cc460 con 0x7fdb54107260 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac63d0 0x56210eac67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210eac7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 0x56210eac80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ea21b50 con 0x56210eac7c20 2023-12-11T19:34:18.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ea1f470 con 0x56210eac63d0 2023-12-11T19:34:18.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ea1ad60 con 0x56210eac6d30 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb5899c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 0x56210eac80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb5899c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 0x56210eac80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48580/0 (socket says 172.21.15.66:48580) 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb5899c700 1 -- 172.21.15.66:0/3049292267 learned_addr learned my addr 172.21.15.66:0/3049292267 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb537fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210eac7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.241+0000 7fdb53fff700 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac63d0 0x56210eac67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009cc0 tx=0x7fdb480116c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 -- 172.21.15.66:0/3049292267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac63d0 msgr2=0x56210eac67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac63d0 0x56210eac67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 -- 172.21.15.66:0/3049292267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 msgr2=0x56210eac80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 0x56210eac80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 -- 172.21.15.66:0/3049292267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56210e897630 con 0x56210eac6d30 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb5899c700 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 0x56210eac80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:18.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210eac7640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb48011ac0 tx=0x7fdb4801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c07234f1921e3833 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb3a7fc700 1 -- 172.21.15.66:0/3049292267 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48009a50 con 0x56210eac6d30 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb3a7fc700 1 -- 172.21.15.66:0/3049292267 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb480142d0 con 0x56210eac6d30 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.242+0000 7fdb3a7fc700 1 -- 172.21.15.66:0/3049292267 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48003ab0 con 0x56210eac6d30 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3049292267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 msgr2=0x56210eac7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210eac7640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb48011ac0 tx=0x7fdb4801c000 comp rx=0 tx=0).stop 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3049292267 shutdown_connections 2023-12-11T19:34:18.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac63d0 0x56210eac67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac7c20 0x56210eac80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3049292267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210eac7640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3049292267 >> 172.21.15.66:0/3049292267 conn(0x56210e9f57e0 msgr2=0x56210eac2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3049292267 shutdown_connections 2023-12-11T19:34:18.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.243+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3049292267 wait complete. 2023-12-11T19:34:18.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.244+0000 7fdb6dcf9b80 1 -- start start 2023-12-11T19:34:18.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 Processor -- start 2023-12-11T19:34:18.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210ead06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210ead7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e9d4560 con 0x56210eac63d0 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e9d4200 con 0x56210eac7c20 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e899920 con 0x56210eac6d30 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.245+0000 7fdb6dcf9b80 10 client.0 _ll_register_callbacks cb 0x56210e8ffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48596/0 (socket says 172.21.15.66:48596) 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 -- 172.21.15.66:0/4198925978 learned_addr learned my addr 172.21.15.66:0/4198925978 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb537fe700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210ead06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb5899c700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210ead7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 -- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 msgr2=0x56210ead06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210ead06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 -- 172.21.15.66:0/4198925978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 msgr2=0x56210ead7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210ead7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 -- 172.21.15.66:0/4198925978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb541a1c10 con 0x56210eac63d0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb53fff700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead01b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400b2e0 tx=0x7fdb44005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9551c80a0330b96 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.246+0000 7fdb337fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb337fe700 1 -- 172.21.15.66:0/4198925978 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4401d070 con 0x56210eac63d0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb337fe700 1 -- 172.21.15.66:0/4198925978 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb4400b710 con 0x56210eac63d0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56210eab9b40 con 0x56210eac63d0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb337fe700 1 -- 172.21.15.66:0/4198925978 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4400b8d0 con 0x56210eac63d0 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb6dcf9b80 10 client.15633 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:18.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fdb48011c70 con 0x56210eac63d0 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb337fe700 1 -- 172.21.15.66:0/4198925978 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56210eab9b40 con 0x56210eac63d0 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.247+0000 7fdb337fe700 1 -- 172.21.15.66:0/4198925978 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb44004bc0 con 0x56210eac63d0 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb6dcf9b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb6dcf9b80 10 client.15633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb6dcf9b80 20 client.15633 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eadbf70 0x56210eade390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb6dcf9b80 4 client.15633 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56210ea95f40 con 0x56210eadbf70 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.248+0000 7fdb537fe700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eadbf70 0x56210eade390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.249+0000 7fdb537fe700 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eadbf70 0x56210eade390 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b83d936e4440f39e server_cookie=c9c2037a99e0105a in_seq=0 out_seq=0 2023-12-11T19:34:18.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.249+0000 7fdb337fe700 10 client.15633 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb337fe700 1 -- 172.21.15.66:0/4198925978 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56210ea95f40 con 0x56210eadbf70 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb337fe700 10 client.15633 handle_command_reply: tid=0 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 client.15633 shutdown 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 msgr2=0x56210ead01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead01b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400b2e0 tx=0x7fdb44005c80 comp rx=0 tx=0).stop 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 shutdown_connections 2023-12-11T19:34:18.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eadbf70 0x56210eade390 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210ead7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead01b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb337fe700 0 client.15633 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/4198925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac6d30 0x56210ead06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 >> 172.21.15.66:0/4198925978 conn(0x56210e9f57e0 msgr2=0x56210eac1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.250+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 shutdown_connections 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.251+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/4198925978 wait complete. 2023-12-11T19:34:18.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.251+0000 7fdb6dcf9b80 20 client.15633 trim_cache size 0 max 16384 2023-12-11T19:34:18.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 Processor -- start 2023-12-11T19:34:18.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 -- start start 2023-12-11T19:34:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac63d0 0x56210eac8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 0x56210ead76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210eace160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ead81a0 con 0x56210eac6d30 2023-12-11T19:34:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210eacb6d0 con 0x56210eac7c20 2023-12-11T19:34:18.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.266+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210eab98d0 con 0x56210eac63d0 2023-12-11T19:34:18.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 0x56210ead76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 0x56210ead76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48606/0 (socket says 172.21.15.66:48606) 2023-12-11T19:34:18.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 -- 172.21.15.66:0/3033366051 learned_addr learned my addr 172.21.15.66:0/3033366051 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb53fff700 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac63d0 0x56210eac8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb5899c700 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210eace160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 -- 172.21.15.66:0/3033366051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac63d0 msgr2=0x56210eac8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac63d0 0x56210eac8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 -- 172.21.15.66:0/3033366051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 msgr2=0x56210eace160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210eace160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 -- 172.21.15.66:0/3033366051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56210e843de0 con 0x56210eac6d30 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb5899c700 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210eace160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.267+0000 7fdb537fe700 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 0x56210ead76f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdb48014440 tx=0x7fdb48005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b1087ede867b12 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb3affd700 1 -- 172.21.15.66:0/3033366051 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48014de0 con 0x56210eac6d30 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb3affd700 1 -- 172.21.15.66:0/3033366051 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb48005550 con 0x56210eac6d30 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb3affd700 1 -- 172.21.15.66:0/3033366051 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480320d0 con 0x56210eac6d30 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3033366051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 msgr2=0x56210ead76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 0x56210ead76f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdb48014440 tx=0x7fdb48005ce0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3033366051 shutdown_connections 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac7c20 0x56210eace160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac63d0 0x56210eac8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/3033366051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac6d30 0x56210ead76f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3033366051 >> 172.21.15.66:0/3033366051 conn(0x56210e9f57e0 msgr2=0x56210eacf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.268+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3033366051 shutdown_connections 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.269+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/3033366051 wait complete. 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.269+0000 7fdb6dcf9b80 1 -- start start 2023-12-11T19:34:18.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.269+0000 7fdb6dcf9b80 1 Processor -- start 2023-12-11T19:34:18.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac6d30 0x56210ead5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac7c20 0x56210ead6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ead6cc0 con 0x56210eac63d0 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ead6e30 con 0x56210eac6d30 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210ead6ff0 con 0x56210eac7c20 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb6dcf9b80 10 client.0 _ll_register_callbacks cb 0x56210e8ffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48608/0 (socket says 172.21.15.66:48608) 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 -- 172.21.15.66:0/2077155202 learned_addr learned my addr 172.21.15.66:0/2077155202 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb537fe700 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac6d30 0x56210ead5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 -- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac7c20 msgr2=0x56210ead6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac7c20 0x56210ead6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 -- 172.21.15.66:0/2077155202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac6d30 msgr2=0x56210ead5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac6d30 0x56210ead5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.270+0000 7fdb53fff700 1 -- 172.21.15.66:0/2077155202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56210e897630 con 0x56210eac63d0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb53fff700 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead5230 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdb44006010 tx=0x7fdb44010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47a14e403fd6bcf server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb337fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb337fe700 1 -- 172.21.15.66:0/2077155202 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4401d070 con 0x56210eac63d0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb480142d0 con 0x56210eac63d0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb337fe700 1 -- 172.21.15.66:0/2077155202 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb44000e40 con 0x56210eac63d0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb337fe700 1 -- 172.21.15.66:0/2077155202 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4400f4b0 con 0x56210eac63d0 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb6dcf9b80 10 client.15639 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:18.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.271+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56210e82cc60 con 0x56210eac63d0 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb337fe700 1 -- 172.21.15.66:0/2077155202 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb44048020 con 0x56210eac63d0 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb337fe700 1 -- 172.21.15.66:0/2077155202 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb440047c0 con 0x56210eac63d0 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb6dcf9b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb6dcf9b80 10 client.15639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb6dcf9b80 20 client.15639 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eb32db0 0x56210eb351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb6dcf9b80 4 client.15639 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56210e9e0fd0 con 0x56210eb32db0 2023-12-11T19:34:18.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.272+0000 7fdb537fe700 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eb32db0 0x56210eb351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.273+0000 7fdb537fe700 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eb32db0 0x56210eb351d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7006847337aadd65 server_cookie=88662783b3c20df7 in_seq=0 out_seq=0 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.273+0000 7fdb337fe700 10 client.15639 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.273+0000 7fdb337fe700 1 -- 172.21.15.66:0/2077155202 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56210e9e0fd0 con 0x56210eb32db0 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.273+0000 7fdb337fe700 10 client.15639 handle_command_reply: tid=0 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.273+0000 7fdb6dcf9b80 1 client.15639 shutdown 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 msgr2=0x56210ead5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead5230 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdb44006010 tx=0x7fdb44010f20 comp rx=0 tx=0).stop 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 shutdown_connections 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x56210eb32db0 0x56210eb351d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56210eac7c20 0x56210ead6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb337fe700 0 client.15639 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:18.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56210eac63d0 0x56210ead5230 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 --2- 172.21.15.66:0/2077155202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56210eac6d30 0x56210ead5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 >> 172.21.15.66:0/2077155202 conn(0x56210e9f57e0 msgr2=0x56210eacab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 shutdown_connections 2023-12-11T19:34:18.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 1 -- 172.21.15.66:0/2077155202 wait complete. 2023-12-11T19:34:18.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.274+0000 7fdb6dcf9b80 20 client.15639 trim_cache size 0 max 16384 2023-12-11T19:34:18.280 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb3c0a1500 msgr2=0x7fdb3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009cc0 tx=0x7fdb480116c0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 msgr2=0x7fdb541004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 0x7fdb541004f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c007ee0 tx=0x7fdb4c00d3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 shutdown_connections 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fdb3c0a1500 0x7fdb3c0a39c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb54102000 0x7fdb54101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb54107260 0x7fdb541004f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 --2- 172.21.15.66:0/3277330811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb54104930 0x7fdb540fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 >> 172.21.15.66:0/3277330811 conn(0x7fdb540fb9f0 msgr2=0x7fdb54103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 shutdown_connections 2023-12-11T19:34:18.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.275+0000 7fdb5a3ff700 1 -- 172.21.15.66:0/3277330811 wait complete. 2023-12-11T19:34:18.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-11T19:34:18.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 -- 172.21.15.66:0/2615858966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 msgr2=0x7f073010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 --2- 172.21.15.66:0/2615858966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f073010b8e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f072800b0a0 tx=0x7f072800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 -- 172.21.15.66:0/2615858966 shutdown_connections 2023-12-11T19:34:18.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 --2- 172.21.15.66:0/2615858966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f073010b8e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 --2- 172.21.15.66:0/2615858966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07300755d0 0x7f0730073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 --2- 172.21.15.66:0/2615858966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0730073ff0 0x7f0730074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 -- 172.21.15.66:0/2615858966 >> 172.21.15.66:0/2615858966 conn(0x7f07300fcb10 msgr2=0x7f07300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 -- 172.21.15.66:0/2615858966 shutdown_connections 2023-12-11T19:34:18.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.526+0000 7f0735dff700 1 -- 172.21.15.66:0/2615858966 wait complete. 2023-12-11T19:34:18.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.527+0000 7f0735dff700 1 Processor -- start 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.527+0000 7f0735dff700 1 -- start start 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f0735dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0730073ff0 0x7f07301996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f0735dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07300755d0 0x7f0730199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f0735dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f07301a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f0735dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07301a1210 con 0x7f0730109450 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f0735dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07301a1350 con 0x7f07300755d0 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f0735dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07301a14e0 con 0x7f0730073ff0 2023-12-11T19:34:18.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f07301a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f07301a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48628/0 (socket says 172.21.15.66:48628) 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 -- 172.21.15.66:0/1965449805 learned_addr learned my addr 172.21.15.66:0/1965449805 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072f7fe700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0730073ff0 0x7f07301996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072effd700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07300755d0 0x7f0730199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 -- 172.21.15.66:0/1965449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0730073ff0 msgr2=0x7f07301996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0730073ff0 0x7f07301996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 -- 172.21.15.66:0/1965449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07300755d0 msgr2=0x7f0730199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07300755d0 0x7f0730199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072ffff700 1 -- 172.21.15.66:0/1965449805 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720009520 con 0x7f0730109450 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.528+0000 7f072effd700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07300755d0 0x7f0730199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.529+0000 7f072ffff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f07301a0c40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f072800b780 tx=0x7f072800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.529+0000 7f072cff9700 1 -- 172.21.15.66:0/1965449805 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800e040 con 0x7f0730109450 2023-12-11T19:34:18.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.529+0000 7f072cff9700 1 -- 172.21.15.66:0/1965449805 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0728003c00 con 0x7f0730109450 2023-12-11T19:34:18.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.529+0000 7f072cff9700 1 -- 172.21.15.66:0/1965449805 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280052a0 con 0x7f0730109450 2023-12-11T19:34:18.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.529+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0728009d00 con 0x7f0730109450 2023-12-11T19:34:18.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.529+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07301a19a0 con 0x7f0730109450 2023-12-11T19:34:18.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.534+0000 7f074974db80 1 Processor -- start 2023-12-11T19:34:18.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.534+0000 7f072cff9700 1 -- 172.21.15.66:0/1965449805 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0728005400 con 0x7f0730109450 2023-12-11T19:34:18.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 -- start start 2023-12-11T19:34:18.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0267363d0 0x55e0267367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026736d30 0x55e026737640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e026737c20 0x55e0267380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e026691b50 con 0x55e0267363d0 2023-12-11T19:34:18.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02668f470 con 0x55e026736d30 2023-12-11T19:34:18.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.535+0000 7f074974db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02668ad60 con 0x55e026737c20 2023-12-11T19:34:18.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0267363d0 0x55e0267367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0267363d0 0x55e0267367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48638/0 (socket says 172.21.15.66:48638) 2023-12-11T19:34:18.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072effd700 1 -- 172.21.15.66:0/900076342 learned_addr learned my addr 172.21.15.66:0/900076342 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072f7fe700 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026736d30 0x55e026737640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072effd700 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e026737c20 0x55e0267380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072f7fe700 1 -- 172.21.15.66:0/900076342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e026737c20 msgr2=0x55e0267380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072f7fe700 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e026737c20 0x55e0267380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072f7fe700 1 -- 172.21.15.66:0/900076342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0267363d0 msgr2=0x55e0267367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072f7fe700 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0267363d0 0x55e0267367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072f7fe700 1 -- 172.21.15.66:0/900076342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e026507630 con 0x55e026736d30 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072effd700 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e026737c20 0x55e0267380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:18.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072cff9700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f07180a1500 0x7f07180a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.536+0000 7f072cff9700 1 -- 172.21.15.66:0/1965449805 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f07280cdd30 con 0x7f0730109450 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.537+0000 7f072f7fe700 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026736d30 0x55e026737640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0720009e40 tx=0x7f0720011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a765417f33d5634 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.537+0000 7f072effd700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f07180a1500 0x7f07180a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.537+0000 7f0715ffb700 1 -- 172.21.15.66:0/900076342 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720019630 con 0x55e026736d30 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.537+0000 7f0715ffb700 1 -- 172.21.15.66:0/900076342 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0720019c20 con 0x55e026736d30 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.537+0000 7f0715ffb700 1 -- 172.21.15.66:0/900076342 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072000ec40 con 0x55e026736d30 2023-12-11T19:34:18.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 -- 172.21.15.66:0/900076342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026736d30 msgr2=0x55e026737640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f072effd700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f07180a1500 0x7f07180a39c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e026737400 tx=0x7f07240098c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026736d30 0x55e026737640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0720009e40 tx=0x7f0720011720 comp rx=0 tx=0).stop 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 -- 172.21.15.66:0/900076342 shutdown_connections 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e026737c20 0x55e0267380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e0267363d0 0x55e0267367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 --2- 172.21.15.66:0/900076342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026736d30 0x55e026737640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 -- 172.21.15.66:0/900076342 >> 172.21.15.66:0/900076342 conn(0x55e0266657e0 msgr2=0x55e026732d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 -- 172.21.15.66:0/900076342 shutdown_connections 2023-12-11T19:34:18.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.538+0000 7f074974db80 1 -- 172.21.15.66:0/900076342 wait complete. 2023-12-11T19:34:18.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.539+0000 7f074974db80 1 -- start start 2023-12-11T19:34:18.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.539+0000 7f074974db80 1 Processor -- start 2023-12-11T19:34:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.539+0000 7f074974db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e0267401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.539+0000 7f074974db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026740710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f074974db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026747750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f074974db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e026644560 con 0x55e026736d30 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f074974db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e026644200 con 0x55e026737c20 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f074974db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e026509920 con 0x55e0267363d0 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f074974db80 10 client.0 _ll_register_callbacks cb 0x55e02656fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e0267401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026740710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e0267401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:49774/0 (socket says 172.21.15.66:49774) 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 -- 172.21.15.66:0/2330832322 learned_addr learned my addr 172.21.15.66:0/2330832322 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072ffff700 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026747750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 -- 172.21.15.66:0/2330832322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 msgr2=0x55e026747750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026747750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 -- 172.21.15.66:0/2330832322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 msgr2=0x55e026740710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026740710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.540+0000 7f072f7fe700 1 -- 172.21.15.66:0/2330832322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0720009a70 con 0x55e0267363d0 2023-12-11T19:34:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.541+0000 7f072f7fe700 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e0267401d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0720009e10 tx=0x7f0720005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4909b4011425d16 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.541+0000 7f070effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:34:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.541+0000 7f070effd700 1 -- 172.21.15.66:0/2330832322 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720004240 con 0x55e0267363d0 2023-12-11T19:34:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.541+0000 7f070effd700 1 -- 172.21.15.66:0/2330832322 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f07200043a0 con 0x55e0267363d0 2023-12-11T19:34:18.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.541+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e026729b40 con 0x55e0267363d0 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.542+0000 7f070effd700 1 -- 172.21.15.66:0/2330832322 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0720004560 con 0x55e0267363d0 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.542+0000 7f074974db80 10 client.25414 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.542+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0264b3de0 con 0x55e0267363d0 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.542+0000 7f070effd700 1 -- 172.21.15.66:0/2330832322 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e026729b40 con 0x55e0267363d0 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.542+0000 7f070effd700 1 -- 172.21.15.66:0/2330832322 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07200039f0 con 0x55e0267363d0 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f074974db80 10 client.25414 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:18.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f074974db80 10 client.25414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:34:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f074974db80 20 client.25414 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f074974db80 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e02674bdb0 0x55e02674e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f074974db80 4 client.25414 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e026705f40 con 0x55e02674bdb0 2023-12-11T19:34:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.543+0000 7f072effd700 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e02674bdb0 0x55e02674e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.544+0000 7f070effd700 10 client.25414 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.544+0000 7f072effd700 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e02674bdb0 0x55e02674e1d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=522c8f030e113ac1 server_cookie=f219c1d529a5d10a in_seq=0 out_seq=0 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f070effd700 1 -- 172.21.15.66:0/2330832322 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e026705f40 con 0x55e02674bdb0 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f070effd700 10 client.25414 handle_command_reply: tid=0 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 client.25414 shutdown 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 msgr2=0x55e0267401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e0267401d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0720009e10 tx=0x7f0720005760 comp rx=0 tx=0).stop 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 shutdown_connections 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e02674bdb0 0x55e02674e1d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026747750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e0267401d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 --2- 172.21.15.66:0/2330832322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026740710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f070effd700 0 client.25414 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 >> 172.21.15.66:0/2330832322 conn(0x55e0266657e0 msgr2=0x55e026731a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 shutdown_connections 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 1 -- 172.21.15.66:0/2330832322 wait complete. 2023-12-11T19:34:18.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.545+0000 7f074974db80 20 client.25414 trim_cache size 0 max 16384 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.560+0000 7f074974db80 1 Processor -- start 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.560+0000 7f074974db80 1 -- start start 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f074974db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026738000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f074974db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026747710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f074974db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e02673e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f074974db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02672db50 con 0x55e026736d30 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f074974db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0267533a0 con 0x55e026737c20 2023-12-11T19:34:18.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f074974db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02673de00 con 0x55e0267363d0 2023-12-11T19:34:18.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f072effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026747710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f072effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026747710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48650/0 (socket says 172.21.15.66:48650) 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f072effd700 1 -- 172.21.15.66:0/2295515980 learned_addr learned my addr 172.21.15.66:0/2295515980 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f072ffff700 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e02673e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.561+0000 7f072f7fe700 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026738000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072effd700 1 -- 172.21.15.66:0/2295515980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 msgr2=0x55e026738000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072effd700 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026738000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072effd700 1 -- 172.21.15.66:0/2295515980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 msgr2=0x55e02673e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072effd700 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e02673e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072effd700 1 -- 172.21.15.66:0/2295515980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02649cc60 con 0x55e026736d30 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072f7fe700 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026738000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f072effd700 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026747710 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55e02674ea90 tx=0x7f0724015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a5d14f1a0ba16c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f07167fc700 1 -- 172.21.15.66:0/2295515980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724024040 con 0x55e026736d30 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f07167fc700 1 -- 172.21.15.66:0/2295515980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07240044c0 con 0x55e026736d30 2023-12-11T19:34:18.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.562+0000 7f07167fc700 1 -- 172.21.15.66:0/2295515980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724003c10 con 0x55e026736d30 2023-12-11T19:34:18.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.563+0000 7f074974db80 1 -- 172.21.15.66:0/2295515980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 msgr2=0x55e026747710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026747710 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55e02674ea90 tx=0x7f0724015040 comp rx=0 tx=0).stop 2023-12-11T19:34:18.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 -- 172.21.15.66:0/2295515980 shutdown_connections 2023-12-11T19:34:18.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e02673e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026738000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 --2- 172.21.15.66:0/2295515980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026736d30 0x55e026747710 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 -- 172.21.15.66:0/2295515980 >> 172.21.15.66:0/2295515980 conn(0x55e0266657e0 msgr2=0x55e02673f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 -- 172.21.15.66:0/2295515980 shutdown_connections 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 -- 172.21.15.66:0/2295515980 wait complete. 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.564+0000 7f074974db80 1 -- start start 2023-12-11T19:34:18.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 Processor -- start 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026745680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026745bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 0x55e026746fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0267474f0 con 0x55e026746b30 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e026747660 con 0x55e026737c20 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e026747820 con 0x55e0267363d0 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f074974db80 10 client.0 _ll_register_callbacks cb 0x55e02656fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f072ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 0x55e026746fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f072ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 0x55e026746fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48652/0 (socket says 172.21.15.66:48652) 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f072ffff700 1 -- 172.21.15.66:0/3493751624 learned_addr learned my addr 172.21.15.66:0/3493751624 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.565+0000 7f072f7fe700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026745680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072effd700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026745bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072ffff700 1 -- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 msgr2=0x55e026745680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072ffff700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026745680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072ffff700 1 -- 172.21.15.66:0/3493751624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 msgr2=0x55e026745bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072ffff700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026745bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072ffff700 1 -- 172.21.15.66:0/3493751624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0724018030 con 0x55e026746b30 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f072ffff700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 0x55e026746fb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f072809ada0 tx=0x7f072800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ef05fd872a03d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.566+0000 7f070effd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f070effd700 1 -- 172.21.15.66:0/3493751624 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072809b7a0 con 0x55e026746b30 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f070effd700 1 -- 172.21.15.66:0/3493751624 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f072809b0a0 con 0x55e026746b30 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0720019c20 con 0x55e026746b30 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f070effd700 1 -- 172.21.15.66:0/3493751624 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f072809f1f0 con 0x55e026746b30 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 10 client.15657 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:18.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0264da250 con 0x55e026746b30 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f070effd700 1 -- 172.21.15.66:0/3493751624 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07280b9020 con 0x55e026746b30 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f070effd700 1 -- 172.21.15.66:0/3493751624 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f072809b300 con 0x55e026746b30 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 10 client.15657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 20 client.15657 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e0267a3520 0x55e0267a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 4 client.15657 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.567+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e026650fd0 con 0x55e0267a3520 2023-12-11T19:34:18.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.568+0000 7f072f7fe700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e0267a3520 0x55e0267a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f070effd700 10 client.15657 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f072f7fe700 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e0267a3520 0x55e0267a5940 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64df5cb2f5021475 server_cookie=40345959d8dbe8d3 in_seq=0 out_seq=0 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f070effd700 1 -- 172.21.15.66:0/3493751624 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e026650fd0 con 0x55e0267a3520 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f070effd700 10 client.15657 handle_command_reply: tid=0 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 client.15657 shutdown 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 msgr2=0x55e026746fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 0x55e026746fb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f072809ada0 tx=0x7f072800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 shutdown_connections 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e026746b30 0x55e026746fb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e0267363d0 0x55e026745680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55e0267a3520 0x55e0267a5940 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 --2- 172.21.15.66:0/3493751624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e026737c20 0x55e026745bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 >> 172.21.15.66:0/3493751624 conn(0x55e0266657e0 msgr2=0x55e02673f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f070effd700 0 client.15657 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:18.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.569+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 shutdown_connections 2023-12-11T19:34:18.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.570+0000 7f074974db80 1 -- 172.21.15.66:0/3493751624 wait complete. 2023-12-11T19:34:18.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.570+0000 7f074974db80 20 client.15657 trim_cache size 0 max 16384 2023-12-11T19:34:18.572 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:18.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.570+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f07180a1500 msgr2=0x7f07180a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.570+0000 7f0735dff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f07180a1500 0x7f07180a39c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e026737400 tx=0x7f07240098c0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.570+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 msgr2=0x7f07301a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.570+0000 7f0735dff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f07301a0c40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f072800b780 tx=0x7f072800bab0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 shutdown_connections 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f07180a1500 0x7f07180a39c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0730109450 0x7f07301a0c40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0730073ff0 0x7f07301996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 --2- 172.21.15.66:0/1965449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07300755d0 0x7f0730199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 >> 172.21.15.66:0/1965449805 conn(0x7f07300fcb10 msgr2=0x7f07301053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 shutdown_connections 2023-12-11T19:34:18.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.571+0000 7f0735dff700 1 -- 172.21.15.66:0/1965449805 wait complete. 2023-12-11T19:34:18.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-11T19:34:18.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.815+0000 7f57df1ff700 1 -- 172.21.15.66:0/458700849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81072e0 msgr2=0x7f57d81096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.815+0000 7f57df1ff700 1 --2- 172.21.15.66:0/458700849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81072e0 0x7f57d81096f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f57d000b0a0 tx=0x7f57d000b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 -- 172.21.15.66:0/458700849 shutdown_connections 2023-12-11T19:34:18.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 --2- 172.21.15.66:0/458700849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d8102080 0x7f57d8104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 --2- 172.21.15.66:0/458700849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d81072e0 0x7f57d81096f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 --2- 172.21.15.66:0/458700849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81049b0 0x7f57d8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 -- 172.21.15.66:0/458700849 >> 172.21.15.66:0/458700849 conn(0x7f57d80fba10 msgr2=0x7f57d80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 -- 172.21.15.66:0/458700849 shutdown_connections 2023-12-11T19:34:18.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.816+0000 7f57df1ff700 1 -- 172.21.15.66:0/458700849 wait complete. 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 Processor -- start 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 -- start start 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 0x7f57d81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81049b0 0x7f57d8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d81072e0 0x7f57d81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81a1220 con 0x7f57d8102080 2023-12-11T19:34:18.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81a1360 con 0x7f57d81072e0 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.817+0000 7f57df1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81a14f0 con 0x7f57d81049b0 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 0x7f57d81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 0x7f57d81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48670/0 (socket says 172.21.15.66:48670) 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 -- 172.21.15.66:0/2622734180 learned_addr learned my addr 172.21.15.66:0/2622734180 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d81072e0 0x7f57d81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57d7fff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81049b0 0x7f57d8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 -- 172.21.15.66:0/2622734180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81049b0 msgr2=0x7f57d8199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81049b0 0x7f57d8199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 -- 172.21.15.66:0/2622734180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d81072e0 msgr2=0x7f57d81a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d81072e0 0x7f57d81a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 -- 172.21.15.66:0/2622734180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0009d00 con 0x7f57d8102080 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.818+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 0x7f57d81996f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00e9a0 tx=0x7f57cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.819+0000 7f57d5ffb700 1 -- 172.21.15.66:0/2622734180 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc009790 con 0x7f57d8102080 2023-12-11T19:34:18.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.819+0000 7f57d5ffb700 1 -- 172.21.15.66:0/2622734180 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57cc00cbd0 con 0x7f57d8102080 2023-12-11T19:34:18.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.819+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d81a17d0 con 0x7f57d8102080 2023-12-11T19:34:18.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.819+0000 7f57d5ffb700 1 -- 172.21.15.66:0/2622734180 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc018680 con 0x7f57d8102080 2023-12-11T19:34:18.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.819+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d81a1d30 con 0x7f57d8102080 2023-12-11T19:34:18.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.822+0000 7f57d5ffb700 1 -- 172.21.15.66:0/2622734180 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f57cc00cd40 con 0x7f57d8102080 2023-12-11T19:34:18.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.823+0000 7f57f2b34b80 1 Processor -- start 2023-12-11T19:34:18.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 -- start start 2023-12-11T19:34:18.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed4557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed456640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed4570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed3b0b50 con 0x562aed456c20 2023-12-11T19:34:18.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed3ae470 con 0x562aed4553d0 2023-12-11T19:34:18.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed3a9d60 con 0x562aed455d30 2023-12-11T19:34:18.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.824+0000 7f57d5ffb700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57c80a1500 0x7f57c80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed456640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57d7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed4570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed456640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:49822/0 (socket says 172.21.15.66:49822) 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dcf9b700 1 -- 172.21.15.66:0/2619503795 learned_addr learned my addr 172.21.15.66:0/2619503795 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed4557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57d5ffb700 1 -- 172.21.15.66:0/2622734180 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57cc014070 con 0x7f57d8102080 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57c80a1500 0x7f57c80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 -- 172.21.15.66:0/2619503795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 msgr2=0x562aed456640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed456640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 -- 172.21.15.66:0/2619503795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 msgr2=0x562aed4570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed4570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57dd79c700 1 -- 172.21.15.66:0/2619503795 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562aed226630 con 0x562aed4553d0 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.825+0000 7f57d7fff700 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed4570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.826+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed4557f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57d000b070 tx=0x7f57d00076f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c13c3473aa674cf server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.826+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57c80a1500 0x7f57c80a39c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57d0024000 tx=0x7f57d0026000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.826+0000 7f57c2ffd700 1 -- 172.21.15.66:0/2619503795 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d000e040 con 0x562aed4553d0 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.826+0000 7f57c2ffd700 1 -- 172.21.15.66:0/2619503795 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d000bcb0 con 0x562aed4553d0 2023-12-11T19:34:18.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.826+0000 7f57c2ffd700 1 -- 172.21.15.66:0/2619503795 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d001d430 con 0x562aed4553d0 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2619503795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 msgr2=0x562aed4557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed4557f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57d000b070 tx=0x7f57d00076f0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2619503795 shutdown_connections 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed4557f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed4570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2619503795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed456640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2619503795 >> 172.21.15.66:0/2619503795 conn(0x562aed3847e0 msgr2=0x562aed451d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2619503795 shutdown_connections 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.827+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2619503795 wait complete. 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.828+0000 7f57f2b34b80 1 -- start start 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.828+0000 7f57f2b34b80 1 Processor -- start 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.828+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed45f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.828+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed45f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.828+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed466780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed363560 con 0x562aed456c20 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed363200 con 0x562aed4553d0 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed228920 con 0x562aed455d30 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57f2b34b80 10 client.0 _ll_register_callbacks cb 0x562aed28ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed466780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed466780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48682/0 (socket says 172.21.15.66:48682) 2023-12-11T19:34:18.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 -- 172.21.15.66:0/2961063471 learned_addr learned my addr 172.21.15.66:0/2961063471 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed45f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57d7fff700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed45f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 -- 172.21.15.66:0/2961063471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 msgr2=0x562aed45f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed45f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 -- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 msgr2=0x562aed45f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed45f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.829+0000 7f57dd79c700 1 -- 172.21.15.66:0/2961063471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c400d7e0 con 0x562aed456c20 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57dd79c700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed466780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57d000d010 tx=0x7f57d00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dad1f01cae9aebb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57b7fff700 1 -- 172.21.15.66:0/2961063471 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d000e070 con 0x562aed456c20 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57b7fff700 1 -- 172.21.15.66:0/2961063471 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57d00052e0 con 0x562aed456c20 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562aed448b40 con 0x562aed456c20 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57b7fff700 1 -- 172.21.15.66:0/2961063471 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d00054a0 con 0x562aed456c20 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57f2b34b80 10 client.15669 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:18.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.830+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f57d810a2e0 con 0x562aed456c20 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57b7fff700 1 -- 172.21.15.66:0/2961063471 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562aed448b40 con 0x562aed456c20 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57b7fff700 1 -- 172.21.15.66:0/2961063471 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57d0030410 con 0x562aed456c20 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57f2b34b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57f2b34b80 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57f2b34b80 20 client.15669 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed46ace0 0x562aed46d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57f2b34b80 4 client.15669 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562aed424f40 con 0x562aed46ace0 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.831+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed46ace0 0x562aed46d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.832+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed46ace0 0x562aed46d100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9978330ae02ffd server_cookie=82a4c5df94e4e2a8 in_seq=0 out_seq=0 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.832+0000 7f57b7fff700 10 client.15669 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57b7fff700 1 -- 172.21.15.66:0/2961063471 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f57cc07ea90 con 0x562aed46ace0 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57b7fff700 10 client.15669 handle_command_reply: tid=0 2023-12-11T19:34:18.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 client.15669 shutdown 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 msgr2=0x562aed466780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed466780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57d000d010 tx=0x7f57d00095a0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 shutdown_connections 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed46ace0 0x562aed46d100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed4553d0 0x562aed45f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed456c20 0x562aed466780 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/2961063471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed45f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 >> 172.21.15.66:0/2961063471 conn(0x562aed3847e0 msgr2=0x562aed450a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.833+0000 7f57b7fff700 0 client.15669 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.834+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 shutdown_connections 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.834+0000 7f57f2b34b80 1 -- 172.21.15.66:0/2961063471 wait complete. 2023-12-11T19:34:18.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.834+0000 7f57f2b34b80 20 client.15669 trim_cache size 0 max 16384 2023-12-11T19:34:18.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 Processor -- start 2023-12-11T19:34:18.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 -- start start 2023-12-11T19:34:18.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed466e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed466740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed456c20 0x562aed45f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed448b40 con 0x562aed4553d0 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed45d9b0 con 0x562aed456c20 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.850+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed45edc0 con 0x562aed455d30 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed466e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed466e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48686/0 (socket says 172.21.15.66:48686) 2023-12-11T19:34:18.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3293569440 learned_addr learned my addr 172.21.15.66:0/3293569440 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dd79c700 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed456c20 0x562aed45f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57d7fff700 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed466740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3293569440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 msgr2=0x562aed466740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed466740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3293569440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed456c20 msgr2=0x562aed45f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed456c20 0x562aed45f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3293569440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562aed1d2de0 con 0x562aed4553d0 2023-12-11T19:34:18.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.851+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed466e30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc06a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeee0ee8e296e99 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57c37fe700 1 -- 172.21.15.66:0/3293569440 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc09e170 con 0x562aed4553d0 2023-12-11T19:34:18.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57c37fe700 1 -- 172.21.15.66:0/3293569440 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562aed424f40 con 0x562aed4553d0 2023-12-11T19:34:18.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57c37fe700 1 -- 172.21.15.66:0/3293569440 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0a2630 con 0x562aed4553d0 2023-12-11T19:34:18.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3293569440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 msgr2=0x562aed466e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed466e30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc06a210 comp rx=0 tx=0).stop 2023-12-11T19:34:18.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3293569440 shutdown_connections 2023-12-11T19:34:18.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed456c20 0x562aed45f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed466e30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3293569440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed455d30 0x562aed466740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.852+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3293569440 >> 172.21.15.66:0/3293569440 conn(0x562aed3847e0 msgr2=0x562aed471a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.853+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3293569440 shutdown_connections 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.853+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3293569440 wait complete. 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.853+0000 7f57f2b34b80 1 -- start start 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.853+0000 7f57f2b34b80 1 Processor -- start 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed464460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed455d30 0x562aed4649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed456c20 0x562aed465870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed465db0 con 0x562aed4553d0 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed465f20 con 0x562aed455d30 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aed4660e0 con 0x562aed456c20 2023-12-11T19:34:18.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57f2b34b80 10 client.0 _ll_register_callbacks cb 0x562aed28ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed464460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57dcf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed464460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48688/0 (socket says 172.21.15.66:48688) 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.854+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3947230817 learned_addr learned my addr 172.21.15.66:0/3947230817 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57d7fff700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed455d30 0x562aed4649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dd79c700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed456c20 0x562aed465870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3947230817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed456c20 msgr2=0x562aed465870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed456c20 0x562aed465870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed455d30 msgr2=0x562aed4649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed455d30 0x562aed4649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dcf9b700 1 -- 172.21.15.66:0/3947230817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d81a1d30 con 0x562aed4553d0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57dcf9b700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed464460 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57cc096730 tx=0x7f57cc09a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6df3bf7b86e673 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57b7fff700 1 -- 172.21.15.66:0/3947230817 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0a2630 con 0x562aed4553d0 2023-12-11T19:34:18.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57d000bcb0 con 0x562aed4553d0 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.855+0000 7f57b7fff700 1 -- 172.21.15.66:0/3947230817 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57cc094e40 con 0x562aed4553d0 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57b7fff700 1 -- 172.21.15.66:0/3947230817 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc09e170 con 0x562aed4553d0 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 10 client.15675 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562aed1bbc60 con 0x562aed4553d0 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57b7fff700 1 -- 172.21.15.66:0/3947230817 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57cc07a020 con 0x562aed4553d0 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57b7fff700 1 -- 172.21.15.66:0/3947230817 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57cc0997b0 con 0x562aed4553d0 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 20 client.15675 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed4c1cd0 0x562aed4c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 4 client.15675 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:18.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.856+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562aed36ffd0 con 0x562aed4c1cd0 2023-12-11T19:34:18.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.857+0000 7f57d7fff700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed4c1cd0 0x562aed4c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:18.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.857+0000 7f57d7fff700 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed4c1cd0 0x562aed4c40f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56aacdd974b72cde server_cookie=22ac6169bbadcbdc in_seq=0 out_seq=0 2023-12-11T19:34:18.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.857+0000 7f57b7fff700 10 client.15675 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:18.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57b7fff700 1 -- 172.21.15.66:0/3947230817 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562aed36ffd0 con 0x562aed4c1cd0 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57b7fff700 10 client.15675 handle_command_reply: tid=0 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 client.15675 shutdown 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 msgr2=0x562aed464460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed464460 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57cc096730 tx=0x7f57cc09a9f0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 shutdown_connections 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562aed4c1cd0 0x562aed4c40f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aed456c20 0x562aed465870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562aed4553d0 0x562aed464460 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 --2- 172.21.15.66:0/3947230817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562aed455d30 0x562aed4649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 >> 172.21.15.66:0/3947230817 conn(0x562aed3847e0 msgr2=0x562aed459c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57b7fff700 0 client.15675 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 shutdown_connections 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 1 -- 172.21.15.66:0/3947230817 wait complete. 2023-12-11T19:34:18.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.858+0000 7f57f2b34b80 20 client.15675 trim_cache size 0 max 16384 2023-12-11T19:34:18.866 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:18.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57c80a1500 msgr2=0x7f57c80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57c80a1500 0x7f57c80a39c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57d0024000 tx=0x7f57d0026000 comp rx=0 tx=0).stop 2023-12-11T19:34:18.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 msgr2=0x7f57d81996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 0x7f57d81996f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00e9a0 tx=0x7f57cc00ecb0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 shutdown_connections 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57c80a1500 0x7f57c80a39c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d81072e0 0x7f57d81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57d8102080 0x7f57d81996f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 --2- 172.21.15.66:0/2622734180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81049b0 0x7f57d8199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 >> 172.21.15.66:0/2622734180 conn(0x7f57d80fba10 msgr2=0x7f57d8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 shutdown_connections 2023-12-11T19:34:18.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:18.859+0000 7f57df1ff700 1 -- 172.21.15.66:0/2622734180 wait complete. 2023-12-11T19:34:19.879 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-11T19:34:19.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-11T19:34:19.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:19 smithi066 ceph-mon[97920]: pgmap v382: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:20.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:19 smithi167 ceph-mon[102750]: pgmap v382: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.099+0000 7fb603fff700 1 -- 172.21.15.66:0/2398211687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc102790 msgr2=0x7fb5fc102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.099+0000 7fb603fff700 1 --2- 172.21.15.66:0/2398211687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc102790 0x7fb5fc102bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0099c0 tx=0x7fb5ec009cd0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 -- 172.21.15.66:0/2398211687 shutdown_connections 2023-12-11T19:34:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 --2- 172.21.15.66:0/2398211687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5fc103130 0x7fb5fc1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 --2- 172.21.15.66:0/2398211687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5fc1015a0 0x7fb5fc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 --2- 172.21.15.66:0/2398211687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc102790 0x7fb5fc102bf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 -- 172.21.15.66:0/2398211687 >> 172.21.15.66:0/2398211687 conn(0x7fb5fc0fcb10 msgr2=0x7fb5fc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 -- 172.21.15.66:0/2398211687 shutdown_connections 2023-12-11T19:34:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.100+0000 7fb603fff700 1 -- 172.21.15.66:0/2398211687 wait complete. 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 Processor -- start 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 -- start start 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5fc1015a0 0x7fb5fc197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5fc102790 0x7fb5fc1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 0x7fb5fc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc19efa0 con 0x7fb5fc103130 2023-12-11T19:34:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc19f0e0 con 0x7fb5fc102790 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.101+0000 7fb603fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc19f270 con 0x7fb5fc1015a0 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 0x7fb5fc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 0x7fb5fc19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48720/0 (socket says 172.21.15.66:48720) 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 -- 172.21.15.66:0/291779976 learned_addr learned my addr 172.21.15.66:0/291779976 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60159a700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5fc102790 0x7fb5fc1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb601d9b700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5fc1015a0 0x7fb5fc197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 -- 172.21.15.66:0/291779976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5fc1015a0 msgr2=0x7fb5fc197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5fc1015a0 0x7fb5fc197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 -- 172.21.15.66:0/291779976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5fc102790 msgr2=0x7fb5fc1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5fc102790 0x7fb5fc1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 -- 172.21.15.66:0/291779976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb5fc103130 2023-12-11T19:34:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.102+0000 7fb60259c700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 0x7fb5fc19e9d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400ef30 tx=0x7fb5f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.103+0000 7fb5f2ffd700 1 -- 172.21.15.66:0/291779976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4018070 con 0x7fb5fc103130 2023-12-11T19:34:20.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.103+0000 7fb5f2ffd700 1 -- 172.21.15.66:0/291779976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f400f040 con 0x7fb5fc103130 2023-12-11T19:34:20.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.103+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc19f4f0 con 0x7fb5fc103130 2023-12-11T19:34:20.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.103+0000 7fb5f2ffd700 1 -- 172.21.15.66:0/291779976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400cbc0 con 0x7fb5fc103130 2023-12-11T19:34:20.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.103+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc19f9b0 con 0x7fb5fc103130 2023-12-11T19:34:20.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.106+0000 7fb5f2ffd700 1 -- 172.21.15.66:0/291779976 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb5f4007500 con 0x7fb5fc103130 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb5f2ffd700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb5e80a1480 0x7fb5e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 Processor -- start 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb5f2ffd700 1 -- 172.21.15.66:0/291779976 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb5f40cc800 con 0x7fb5fc103130 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb601d9b700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb5e80a1480 0x7fb5e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 -- start start 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x562080400870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x5620804016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 0x562080402120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56208035bbd0 con 0x562080401ca0 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620803594f0 con 0x562080400db0 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.108+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080354de0 con 0x562080400450 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb601d9b700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb5e80a1480 0x7fb5e80a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc1025f0 tx=0x7fb5f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 0x562080402120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 0x562080402120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48726/0 (socket says 172.21.15.66:48726) 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 -- 172.21.15.66:0/141187942 learned_addr learned my addr 172.21.15.66:0/141187942 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb601d9b700 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x562080400870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60159a700 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x5620804016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 -- 172.21.15.66:0/141187942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 msgr2=0x562080400870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x562080400870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 -- 172.21.15.66:0/141187942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 msgr2=0x5620804016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x5620804016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 -- 172.21.15.66:0/141187942 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620801d16b0 con 0x562080401ca0 2023-12-11T19:34:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.109+0000 7fb60259c700 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 0x562080402120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40965e0 tx=0x7fb5f4099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577aba7696e5ea15 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb5e7fff700 1 -- 172.21.15.66:0/141187942 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4012340 con 0x562080401ca0 2023-12-11T19:34:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb5e7fff700 1 -- 172.21.15.66:0/141187942 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f409a270 con 0x562080401ca0 2023-12-11T19:34:20.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb5e7fff700 1 -- 172.21.15.66:0/141187942 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400cd80 con 0x562080401ca0 2023-12-11T19:34:20.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:19 smithi070 ceph-mon[104044]: pgmap v382: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 -- 172.21.15.66:0/141187942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 msgr2=0x562080402120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 0x562080402120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40965e0 tx=0x7fb5f4099730 comp rx=0 tx=0).stop 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 -- 172.21.15.66:0/141187942 shutdown_connections 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080401ca0 0x562080402120 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x562080400870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/141187942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x5620804016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 -- 172.21.15.66:0/141187942 >> 172.21.15.66:0/141187942 conn(0x56208032f860 msgr2=0x5620803fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.110+0000 7fb6178cbb80 1 -- 172.21.15.66:0/141187942 shutdown_connections 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.111+0000 7fb6178cbb80 1 -- 172.21.15.66:0/141187942 wait complete. 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.111+0000 7fb6178cbb80 1 -- start start 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.111+0000 7fb6178cbb80 1 Processor -- start 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 0x56208040e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x56208040ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56208030e5e0 con 0x562080400450 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56208030e280 con 0x562080400db0 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620801d39a0 con 0x562080401ca0 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb6178cbb80 10 client.0 _ll_register_callbacks cb 0x562080239b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 0x56208040e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 0x56208040e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48742/0 (socket says 172.21.15.66:48742) 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb601d9b700 1 -- 172.21.15.66:0/765595659 learned_addr learned my addr 172.21.15.66:0/765595659 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb60259c700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb60159a700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x56208040ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.112+0000 7fb601d9b700 1 -- 172.21.15.66:0/765595659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 msgr2=0x562080415bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb601d9b700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb601d9b700 1 -- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 msgr2=0x56208040ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb601d9b700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x56208040ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb601d9b700 1 -- 172.21.15.66:0/765595659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc19f9b0 con 0x562080400450 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb601d9b700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 0x56208040e6c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80116c0 tx=0x7fb5f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf3b7c84b822b90 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb5e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb5e57fa700 1 -- 172.21.15.66:0/765595659 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f801e830 con 0x562080400450 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620803f3bc0 con 0x562080400450 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb5e57fa700 1 -- 172.21.15.66:0/765595659 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb5f8017070 con 0x562080400450 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb5e57fa700 1 -- 172.21.15.66:0/765595659 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5f8011e60 con 0x562080400450 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb6178cbb80 10 client.15687 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.113+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56208017de60 con 0x562080400450 2023-12-11T19:34:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb5e57fa700 1 -- 172.21.15.66:0/765595659 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5620803f3bc0 con 0x562080400450 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb5e57fa700 1 -- 172.21.15.66:0/765595659 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb5f801c9c0 con 0x562080400450 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb6178cbb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb6178cbb80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb6178cbb80 20 client.15687 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56208041a4f0 0x56208041c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb6178cbb80 4 client.15687 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.114+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620803cffc0 con 0x56208041a4f0 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.115+0000 7fb60159a700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56208041a4f0 0x56208041c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.115+0000 7fb60159a700 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56208041a4f0 0x56208041c910 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783f3cfc1e7e586e server_cookie=62e6ab99207e92eb in_seq=0 out_seq=0 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.115+0000 7fb5e57fa700 10 client.15687 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb5e57fa700 1 -- 172.21.15.66:0/765595659 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620803cffc0 con 0x56208041a4f0 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb5e57fa700 10 client.15687 handle_command_reply: tid=0 2023-12-11T19:34:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 client.15687 shutdown 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 msgr2=0x56208040e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 0x56208040e6c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80116c0 tx=0x7fb5f80047c0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 shutdown_connections 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56208041a4f0 0x56208041c910 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb5e57fa700 0 client.15687 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400450 0x56208040e6c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.116+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/765595659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400db0 0x56208040ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.117+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 >> 172.21.15.66:0/765595659 conn(0x56208032f860 msgr2=0x5620803fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.117+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 shutdown_connections 2023-12-11T19:34:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.117+0000 7fb6178cbb80 1 -- 172.21.15.66:0/765595659 wait complete. 2023-12-11T19:34:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.117+0000 7fb6178cbb80 20 client.15687 trim_cache size 0 max 16384 2023-12-11T19:34:20.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.141+0000 7fb6178cbb80 1 Processor -- start 2023-12-11T19:34:20.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.141+0000 7fb6178cbb80 1 -- start start 2023-12-11T19:34:20.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x56208040cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080415bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 0x56208040d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620804011d0 con 0x562080400db0 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080416da0 con 0x562080401ca0 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56208040d690 con 0x562080400450 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.142+0000 7fb60259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 0x56208040d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x56208040cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 0x56208040d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:51902/0 (socket says 172.21.15.66:51902) 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 -- 172.21.15.66:0/3794484652 learned_addr learned my addr 172.21.15.66:0/3794484652 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60159a700 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080415bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 -- 172.21.15.66:0/3794484652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 msgr2=0x56208040cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x56208040cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 -- 172.21.15.66:0/3794484652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 msgr2=0x562080415bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080415bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 -- 172.21.15.66:0/3794484652 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562080166ce0 con 0x562080401ca0 2023-12-11T19:34:20.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb601d9b700 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x56208040cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60159a700 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080415bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.143+0000 7fb60259c700 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 0x56208040d150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4096070 tx=0x7fb5f400c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a429159879effd29 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.144+0000 7fb5f0ff9700 1 -- 172.21.15.66:0/3794484652 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f409aea0 con 0x562080401ca0 2023-12-11T19:34:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.144+0000 7fb5f0ff9700 1 -- 172.21.15.66:0/3794484652 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f40a1e70 con 0x562080401ca0 2023-12-11T19:34:20.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.144+0000 7fb5f0ff9700 1 -- 172.21.15.66:0/3794484652 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4097fa0 con 0x562080401ca0 2023-12-11T19:34:20.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3794484652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 msgr2=0x56208040d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 0x56208040d150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4096070 tx=0x7fb5f400c7a0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3794484652 shutdown_connections 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080401ca0 0x56208040d150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080400450 0x56208040cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3794484652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080415bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3794484652 >> 172.21.15.66:0/3794484652 conn(0x56208032f860 msgr2=0x562080409620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3794484652 shutdown_connections 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3794484652 wait complete. 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 -- start start 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.145+0000 7fb6178cbb80 1 Processor -- start 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400450 0x562080411cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080412230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080415740 con 0x562080400db0 2023-12-11T19:34:20.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620804158b0 con 0x562080400450 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562080415a70 con 0x562080401ca0 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb6178cbb80 10 client.0 _ll_register_callbacks cb 0x562080239b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb60159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080412230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb60159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080412230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48750/0 (socket says 172.21.15.66:48750) 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb60159a700 1 -- 172.21.15.66:0/3660361609 learned_addr learned my addr 172.21.15.66:0/3660361609 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.146+0000 7fb601d9b700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400450 0x562080411cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60259c700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60159a700 1 -- 172.21.15.66:0/3660361609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 msgr2=0x562080415200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60159a700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60159a700 1 -- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400450 msgr2=0x562080411cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60159a700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400450 0x562080411cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60159a700 1 -- 172.21.15.66:0/3660361609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56208017de60 con 0x562080400db0 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60259c700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:20.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb601d9b700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400450 0x562080411cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb60159a700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080412230 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009990 tx=0x7fb5ec00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e83250c4debc16 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb5e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.147+0000 7fb5e57fa700 1 -- 172.21.15.66:0/3660361609 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0054c0 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5f409a270 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb5e57fa700 1 -- 172.21.15.66:0/3660361609 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb5ec005620 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb5e57fa700 1 -- 172.21.15.66:0/3660361609 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5ec003760 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 10 client.15690 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5620801d16b0 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb5e57fa700 1 -- 172.21.15.66:0/3660361609 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5f409a270 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb5e57fa700 1 -- 172.21.15.66:0/3660361609 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb5ec0274f0 con 0x562080400db0 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 20 client.15690 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:20.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562080471de0 0x562080474200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 4 client.15690 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-11T19:34:20.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.148+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56208031b050 con 0x562080471de0 2023-12-11T19:34:20.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.149+0000 7fb601d9b700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562080471de0 0x562080474200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.150+0000 7fb601d9b700 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562080471de0 0x562080474200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6601f6a9fdcbfc05 server_cookie=57b75cd8af85424d in_seq=0 out_seq=0 2023-12-11T19:34:20.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.150+0000 7fb5e57fa700 10 client.15690 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb5e57fa700 1 -- 172.21.15.66:0/3660361609 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56208031b050 con 0x562080471de0 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb5e57fa700 10 client.15690 handle_command_reply: tid=0 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 client.15690 shutdown 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 msgr2=0x562080412230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080412230 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009990 tx=0x7fb5ec00bac0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 shutdown_connections 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x562080471de0 0x562080474200 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562080400450 0x562080411cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562080401ca0 0x562080415200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb5e57fa700 0 client.15690 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 --2- 172.21.15.66:0/3660361609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562080400db0 0x562080412230 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.151+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 >> 172.21.15.66:0/3660361609 conn(0x56208032f860 msgr2=0x56208040e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 shutdown_connections 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb6178cbb80 1 -- 172.21.15.66:0/3660361609 wait complete. 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb6178cbb80 20 client.15690 trim_cache size 0 max 16384 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "518a99e2-89d5-45bc-a3f2-a2d99d9ecd49", 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-12-11T19:34:20.155 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb5e80a1480 msgr2=0x7fb5e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb5e80a1480 0x7fb5e80a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc1025f0 tx=0x7fb5f800b410 comp rx=0 tx=0).stop 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 msgr2=0x7fb5fc19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 0x7fb5fc19e9d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400ef30 tx=0x7fb5f400c3a0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 shutdown_connections 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb5e80a1480 0x7fb5e80a3940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5fc103130 0x7fb5fc19e9d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5fc1015a0 0x7fb5fc197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 --2- 172.21.15.66:0/291779976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5fc102790 0x7fb5fc1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.152+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 >> 172.21.15.66:0/291779976 conn(0x7fb5fc0fcb10 msgr2=0x7fb5fc105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.153+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 shutdown_connections 2023-12-11T19:34:20.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.153+0000 7fb603fff700 1 -- 172.21.15.66:0/291779976 wait complete. 2023-12-11T19:34:20.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:34:20.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 -- 172.21.15.66:0/1828799296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 msgr2=0x7f9940100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 --2- 172.21.15.66:0/1828799296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940100290 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9930009990 tx=0x7f9930009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 -- 172.21.15.66:0/1828799296 shutdown_connections 2023-12-11T19:34:20.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 --2- 172.21.15.66:0/1828799296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9940101190 0x7f9940101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 --2- 172.21.15.66:0/1828799296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940100290 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 --2- 172.21.15.66:0/1828799296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99401007d0 0x7f9940100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 -- 172.21.15.66:0/1828799296 >> 172.21.15.66:0/1828799296 conn(0x7f99400fb9f0 msgr2=0x7f99400fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 -- 172.21.15.66:0/1828799296 shutdown_connections 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.387+0000 7f9945fff700 1 -- 172.21.15.66:0/1828799296 wait complete. 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.388+0000 7f9945fff700 1 Processor -- start 2023-12-11T19:34:20.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.388+0000 7f9945fff700 1 -- start start 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f9945fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f9945fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99401007d0 0x7f9940199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f9945fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9940101190 0x7f99401a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f9945fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401a1190 con 0x7f99400ffe70 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f9945fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401a12d0 con 0x7f9940101190 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f9945fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401a1460 con 0x7f99401007d0 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f993f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f993f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48762/0 (socket says 172.21.15.66:48762) 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f993f7fe700 1 -- 172.21.15.66:0/3711016504 learned_addr learned my addr 172.21.15.66:0/3711016504 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f993effd700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99401007d0 0x7f9940199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.389+0000 7f993ffff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9940101190 0x7f99401a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993f7fe700 1 -- 172.21.15.66:0/3711016504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99401007d0 msgr2=0x7f9940199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99401007d0 0x7f9940199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993f7fe700 1 -- 172.21.15.66:0/3711016504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9940101190 msgr2=0x7f99401a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9940101190 0x7f99401a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993f7fe700 1 -- 172.21.15.66:0/3711016504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300095f0 con 0x7f99400ffe70 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993ffff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9940101190 0x7f99401a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940199660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9930006010 tx=0x7f9930004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993cff9700 1 -- 172.21.15.66:0/3711016504 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993001d070 con 0x7f99400ffe70 2023-12-11T19:34:20.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993cff9700 1 -- 172.21.15.66:0/3711016504 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993000bd50 con 0x7f99400ffe70 2023-12-11T19:34:20.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f993cff9700 1 -- 172.21.15.66:0/3711016504 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000fd90 con 0x7f99400ffe70 2023-12-11T19:34:20.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.390+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401a16e0 con 0x7f99400ffe70 2023-12-11T19:34:20.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.391+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401a1ab0 con 0x7f99400ffe70 2023-12-11T19:34:20.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.394+0000 7f993cff9700 1 -- 172.21.15.66:0/3711016504 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f993000f4f0 con 0x7f99400ffe70 2023-12-11T19:34:20.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.395+0000 7f993cff9700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99280a1480 0x7f99280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.395+0000 7f993cff9700 1 -- 172.21.15.66:0/3711016504 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99300ce050 con 0x7f99400ffe70 2023-12-11T19:34:20.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.396+0000 7f993effd700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99280a1480 0x7f99280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.396+0000 7f99598aab80 1 Processor -- start 2023-12-11T19:34:20.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.396+0000 7f99598aab80 1 -- start start 2023-12-11T19:34:20.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.396+0000 7f993effd700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99280a1480 0x7f99280a3940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9934009c60 tx=0x7f9934009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f99598aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 0x5578493747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f99598aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x557849375640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f99598aab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f99598aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578492cfb50 con 0x5578493743d0 2023-12-11T19:34:20.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f99598aab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578492cd470 con 0x557849375c20 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f99598aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578492c8d60 con 0x557849374d30 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 0x5578493747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 0x5578493747f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48778/0 (socket says 172.21.15.66:48778) 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 -- 172.21.15.66:0/2571780265 learned_addr learned my addr 172.21.15.66:0/2571780265 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993ffff700 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993effd700 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x557849375640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 -- 172.21.15.66:0/2571780265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 msgr2=0x557849375640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x557849375640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 -- 172.21.15.66:0/2571780265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 msgr2=0x5578493760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993f7fe700 1 -- 172.21.15.66:0/2571780265 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557849145630 con 0x5578493743d0 2023-12-11T19:34:20.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.397+0000 7f993effd700 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x557849375640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:20.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.398+0000 7f993f7fe700 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 0x5578493747f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f993009b100 tx=0x7f9930005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368ea1f1421a1f92 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.398+0000 7f9925ffb700 1 -- 172.21.15.66:0/2571780265 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993009ad60 con 0x5578493743d0 2023-12-11T19:34:20.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.398+0000 7f9925ffb700 1 -- 172.21.15.66:0/2571780265 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993009bb00 con 0x5578493743d0 2023-12-11T19:34:20.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.398+0000 7f9925ffb700 1 -- 172.21.15.66:0/2571780265 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993009f320 con 0x5578493743d0 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 -- 172.21.15.66:0/2571780265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 msgr2=0x5578493747f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 0x5578493747f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f993009b100 tx=0x7f9930005910 comp rx=0 tx=0).stop 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 -- 172.21.15.66:0/2571780265 shutdown_connections 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493760a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5578493743d0 0x5578493747f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 --2- 172.21.15.66:0/2571780265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x557849375640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 -- 172.21.15.66:0/2571780265 >> 172.21.15.66:0/2571780265 conn(0x5578492a37e0 msgr2=0x557849370d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 -- 172.21.15.66:0/2571780265 shutdown_connections 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.399+0000 7f99598aab80 1 -- 172.21.15.66:0/2571780265 wait complete. 2023-12-11T19:34:20.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 -- start start 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 Processor -- start 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5578493743d0 0x55784937deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x55784937e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849385430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557849282560 con 0x557849375c20 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557849282200 con 0x5578493743d0 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557849147920 con 0x557849374d30 2023-12-11T19:34:20.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.400+0000 7f99598aab80 10 client.0 _ll_register_callbacks cb 0x5578491ada80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x55784937e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849385430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x55784937e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:49910/0 (socket says 172.21.15.66:49910) 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 -- 172.21.15.66:0/3185441892 learned_addr learned my addr 172.21.15.66:0/3185441892 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5578493743d0 0x55784937deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 -- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5578493743d0 msgr2=0x55784937deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5578493743d0 0x55784937deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 -- 172.21.15.66:0/3185441892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 msgr2=0x557849385430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849385430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993effd700 1 -- 172.21.15.66:0/3185441892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401a1ab0 con 0x557849374d30 2023-12-11T19:34:20.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.401+0000 7f993ffff700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849385430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f993effd700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x55784937e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9934003980 tx=0x7f99340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78784d2c6279a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f991effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f991effd700 1 -- 172.21.15.66:0/3185441892 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993401e8b0 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557849367b40 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f991effd700 1 -- 172.21.15.66:0/3185441892 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f993401ea10 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f991effd700 1 -- 172.21.15.66:0/3185441892 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f993401ebd0 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f99598aab80 10 client.25465 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.402+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5578490f1de0 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f991effd700 1 -- 172.21.15.66:0/3185441892 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557849367b40 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f991effd700 1 -- 172.21.15.66:0/3185441892 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9934031400 con 0x557849374d30 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f99598aab80 10 client.25465 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:20.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f99598aab80 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f99598aab80 20 client.25465 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f99598aab80 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557849389cc0 0x55784938c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f99598aab80 4 client.25465 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557849343f40 con 0x557849389cc0 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.403+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557849389cc0 0x55784938c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.404+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557849389cc0 0x55784938c0e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2c29591bed2200 server_cookie=ff242f75c4c2f1f5 in_seq=0 out_seq=0 2023-12-11T19:34:20.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.404+0000 7f991effd700 10 client.25465 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f991effd700 1 -- 172.21.15.66:0/3185441892 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557849343f40 con 0x557849389cc0 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f991effd700 10 client.25465 handle_command_reply: tid=0 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 client.25465 shutdown 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 msgr2=0x55784937e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x55784937e3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9934003980 tx=0x7f99340047c0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 shutdown_connections 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557849389cc0 0x55784938c0e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5578493743d0 0x55784937deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f991effd700 0 client.25465 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849385430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 --2- 172.21.15.66:0/3185441892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557849374d30 0x55784937e3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 >> 172.21.15.66:0/3185441892 conn(0x5578492a37e0 msgr2=0x55784936fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 shutdown_connections 2023-12-11T19:34:20.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 1 -- 172.21.15.66:0/3185441892 wait complete. 2023-12-11T19:34:20.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.405+0000 7f99598aab80 20 client.25465 trim_cache size 0 max 16384 2023-12-11T19:34:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.423+0000 7f99598aab80 1 Processor -- start 2023-12-11T19:34:20.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.423+0000 7f99598aab80 1 -- start start 2023-12-11T19:34:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.423+0000 7f99598aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849376000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f99598aab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849374d30 0x5578493853f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f99598aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849378b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f99598aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578493912f0 con 0x557849375c20 2023-12-11T19:34:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f99598aab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784936bb50 con 0x557849374d30 2023-12-11T19:34:20.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f99598aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784937eb30 con 0x5578493743d0 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849378b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849378b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48802/0 (socket says 172.21.15.66:48802) 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 -- 172.21.15.66:0/3028297986 learned_addr learned my addr 172.21.15.66:0/3028297986 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993f7fe700 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849376000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993effd700 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849374d30 0x5578493853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 -- 172.21.15.66:0/3028297986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 msgr2=0x557849376000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849376000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 -- 172.21.15.66:0/3028297986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849374d30 msgr2=0x5578493853f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849374d30 0x5578493853f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.424+0000 7f993ffff700 1 -- 172.21.15.66:0/3028297986 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578490dac60 con 0x557849375c20 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.425+0000 7f993effd700 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849374d30 0x5578493853f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.425+0000 7f993ffff700 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849378b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9938007900 tx=0x7f993800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e352fc3d2d13dbf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.425+0000 7f99267fc700 1 -- 172.21.15.66:0/3028297986 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800e040 con 0x557849375c20 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.425+0000 7f99267fc700 1 -- 172.21.15.66:0/3028297986 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993800f040 con 0x557849375c20 2023-12-11T19:34:20.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.425+0000 7f99267fc700 1 -- 172.21.15.66:0/3028297986 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938004280 con 0x557849375c20 2023-12-11T19:34:20.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 -- 172.21.15.66:0/3028297986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 msgr2=0x557849378b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849378b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9938007900 tx=0x7f993800b040 comp rx=0 tx=0).stop 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 -- 172.21.15.66:0/3028297986 shutdown_connections 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849375c20 0x557849378b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849376000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 --2- 172.21.15.66:0/3028297986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849374d30 0x5578493853f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 -- 172.21.15.66:0/3028297986 >> 172.21.15.66:0/3028297986 conn(0x5578492a37e0 msgr2=0x55784937c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 -- 172.21.15.66:0/3028297986 shutdown_connections 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.426+0000 7f99598aab80 1 -- 172.21.15.66:0/3028297986 wait complete. 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 -- start start 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 Processor -- start 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849381090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 0x5578493815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557849384b30 con 0x557849374d30 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557849384ca0 con 0x557849375c20 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557849384e60 con 0x5578493743d0 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.427+0000 7f99598aab80 10 client.0 _ll_register_callbacks cb 0x5578491ada80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 0x5578493815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 0x5578493815d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48818/0 (socket says 172.21.15.66:48818) 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 -- 172.21.15.66:0/2844159710 learned_addr learned my addr 172.21.15.66:0/2844159710 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849381090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993ffff700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 -- 172.21.15.66:0/2844159710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 msgr2=0x557849381090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849381090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 -- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 msgr2=0x5578493845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 -- 172.21.15.66:0/2844159710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99380075b0 con 0x557849374d30 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993f7fe700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849381090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.428+0000 7f993effd700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 0x5578493815d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9934011b30 tx=0x7f99340046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52abe5655c149b3d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f991effd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f991effd700 1 -- 172.21.15.66:0/2844159710 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993401e6b0 con 0x557849374d30 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f993009bb00 con 0x557849374d30 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f991effd700 1 -- 172.21.15.66:0/2844159710 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f993402d030 con 0x557849374d30 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f991effd700 1 -- 172.21.15.66:0/2844159710 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f993401e810 con 0x557849374d30 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f99598aab80 10 client.15708 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:20.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557849118250 con 0x557849374d30 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.429+0000 7f991effd700 1 -- 172.21.15.66:0/2844159710 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f993009bb00 con 0x557849374d30 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f991effd700 1 -- 172.21.15.66:0/2844159710 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9934027880 con 0x557849374d30 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f99598aab80 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f99598aab80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f99598aab80 20 client.15708 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f99598aab80 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5578493e0f70 0x5578493e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f99598aab80 4 client.15708 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55784928efd0 con 0x5578493e0f70 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.430+0000 7f993f7fe700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5578493e0f70 0x5578493e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f993f7fe700 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5578493e0f70 0x5578493e3390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c13daeac6cb98e server_cookie=d7a4fb8bc26cc828 in_seq=0 out_seq=0 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f991effd700 10 client.15708 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f991effd700 1 -- 172.21.15.66:0/2844159710 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55784928efd0 con 0x5578493e0f70 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f991effd700 10 client.15708 handle_command_reply: tid=0 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 client.15708 shutdown 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 msgr2=0x5578493815d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 0x5578493815d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9934011b30 tx=0x7f99340046e0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 shutdown_connections 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5578493e0f70 0x5578493e3390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557849375c20 0x5578493845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578493743d0 0x557849381090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 --2- 172.21.15.66:0/2844159710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557849374d30 0x5578493815d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 >> 172.21.15.66:0/2844159710 conn(0x5578492a37e0 msgr2=0x55784937de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f991effd700 0 client.15708 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 shutdown_connections 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.431+0000 7f99598aab80 1 -- 172.21.15.66:0/2844159710 wait complete. 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.432+0000 7f99598aab80 20 client.15708 trim_cache size 0 max 16384 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stdout: "518a99e2-89d5-45bc-a3f2-a2d99d9ecd49": { 2023-12-11T19:34:20.435 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-12-11T19:34:20.436 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "518a99e2-89d5-45bc-a3f2-a2d99d9ecd49", 2023-12-11T19:34:20.436 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-12-11T19:34:20.436 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:34:20.436 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:34:20.436 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.432+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99280a1480 msgr2=0x7f99280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.432+0000 7f9945fff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99280a1480 0x7f99280a3940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9934009c60 tx=0x7f9934009260 comp rx=0 tx=0).stop 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.432+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 msgr2=0x7f9940199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.432+0000 7f9945fff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940199660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9930006010 tx=0x7f9930004730 comp rx=0 tx=0).stop 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 shutdown_connections 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f99280a1480 0x7f99280a3940 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9940101190 0x7f99401a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99400ffe70 0x7f9940199660 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 --2- 172.21.15.66:0/3711016504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99401007d0 0x7f9940199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 >> 172.21.15.66:0/3711016504 conn(0x7f99400fb9f0 msgr2=0x7f99400fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:20.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 shutdown_connections 2023-12-11T19:34:20.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:20.433+0000 7f9945fff700 1 -- 172.21.15.66:0/3711016504 wait complete. 2023-12-11T19:34:20.457 INFO:tasks.cephfs.filesystem:scrub status for tag:518a99e2-89d5-45bc-a3f2-a2d99d9ecd49 - {'path': '/', 'tag': '518a99e2-89d5-45bc-a3f2-a2d99d9ecd49', 'options': 'recursive,force'} 2023-12-11T19:34:22.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:21 smithi167 ceph-mon[102750]: scrub queued for path: / 2023-12-11T19:34:22.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:21 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:22.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:21 smithi167 ceph-mon[102750]: scrub summary: active paths [/] 2023-12-11T19:34:22.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:21 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:22.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:21 smithi167 ceph-mon[102750]: pgmap v383: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:22.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:21 smithi167 ceph-mon[102750]: scrub summary: idle 2023-12-11T19:34:22.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:21 smithi066 ceph-mon[97920]: scrub queued for path: / 2023-12-11T19:34:22.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:21 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:22.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:21 smithi066 ceph-mon[97920]: scrub summary: active paths [/] 2023-12-11T19:34:22.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:21 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:22.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:21 smithi066 ceph-mon[97920]: pgmap v383: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:22.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:21 smithi066 ceph-mon[97920]: scrub summary: idle 2023-12-11T19:34:22.117 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:21 smithi070 ceph-mon[104044]: scrub queued for path: / 2023-12-11T19:34:22.117 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:21 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:22.117 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:21 smithi070 ceph-mon[104044]: scrub summary: active paths [/] 2023-12-11T19:34:22.117 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:21 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:22.117 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:21 smithi070 ceph-mon[104044]: pgmap v383: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:22.117 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:21 smithi070 ceph-mon[104044]: scrub summary: idle 2023-12-11T19:34:24.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:23 smithi167 ceph-mon[102750]: pgmap v384: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:24.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:23 smithi066 ceph-mon[97920]: pgmap v384: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:24.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:23 smithi070 ceph-mon[104044]: pgmap v384: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 -- 172.21.15.66:0/3413749241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 msgr2=0x7f25f8074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/3413749241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8074450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009990 tx=0x7f25e8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 -- 172.21.15.66:0/3413749241 shutdown_connections 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/3413749241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f80755b0 0x7f25f8073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/3413749241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f8109420 0x7f25f810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/3413749241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8074450 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 -- 172.21.15.66:0/3413749241 >> 172.21.15.66:0/3413749241 conn(0x7f25f80fcac0 msgr2=0x7f25f80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 -- 172.21.15.66:0/3413749241 shutdown_connections 2023-12-11T19:34:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.627+0000 7f25ff7ff700 1 -- 172.21.15.66:0/3413749241 wait complete. 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.628+0000 7f25ff7ff700 1 Processor -- start 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 -- start start 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f80755b0 0x7f25f8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8109420 0x7f25f81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81a10c0 con 0x7f25f8073fd0 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81a1200 con 0x7f25f8109420 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25ff7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81a1390 con 0x7f25f80755b0 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25fd59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25fd59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57980/0 (socket says 172.21.15.66:57980) 2023-12-11T19:34:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25fd59b700 1 -- 172.21.15.66:0/1824540505 learned_addr learned my addr 172.21.15.66:0/1824540505 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.629+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8109420 0x7f25f81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f80755b0 0x7f25f8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fd59b700 1 -- 172.21.15.66:0/1824540505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f80755b0 msgr2=0x7f25f8199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fd59b700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f80755b0 0x7f25f8199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fd59b700 1 -- 172.21.15.66:0/1824540505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8109420 msgr2=0x7f25f81a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fd59b700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8109420 0x7f25f81a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fd59b700 1 -- 172.21.15.66:0/1824540505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80095f0 con 0x7f25f8073fd0 2023-12-11T19:34:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25fd59b700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8199590 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f25f400d980 tx=0x7f25f400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25ee7fc700 1 -- 172.21.15.66:0/1824540505 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f400bb60 con 0x7f25f8073fd0 2023-12-11T19:34:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25ee7fc700 1 -- 172.21.15.66:0/1824540505 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25f4007640 con 0x7f25f8073fd0 2023-12-11T19:34:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f81a1670 con 0x7f25f8073fd0 2023-12-11T19:34:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.630+0000 7f25ee7fc700 1 -- 172.21.15.66:0/1824540505 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4004d60 con 0x7f25f8073fd0 2023-12-11T19:34:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.631+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f81a1bd0 con 0x7f25f8073fd0 2023-12-11T19:34:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.635+0000 7f25ee7fc700 1 -- 172.21.15.66:0/1824540505 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f25f4004020 con 0x7f25f8073fd0 2023-12-11T19:34:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 Processor -- start 2023-12-11T19:34:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 -- start start 2023-12-11T19:34:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b8bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288ae5af0 con 0x55e288b8a830 2023-12-11T19:34:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288ae3410 con 0x55e288b89ed0 2023-12-11T19:34:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f26131beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288aded00 con 0x55e288b8b720 2023-12-11T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.637+0000 7f25fd59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b8b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57982/0 (socket says 172.21.15.66:57982) 2023-12-11T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 -- 172.21.15.66:0/2255839676 learned_addr learned my addr 172.21.15.66:0/2255839676 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b8bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 -- 172.21.15.66:0/2255839676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 msgr2=0x55e288b8bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b8bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 -- 172.21.15.66:0/2255839676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 msgr2=0x55e288b8a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fd59b700 1 -- 172.21.15.66:0/2255839676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e28895b5d0 con 0x55e288b8a830 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25ee7fc700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25e40a1500 0x7f25e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25ee7fc700 1 -- 172.21.15.66:0/1824540505 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25f40ccbf0 con 0x7f25f8073fd0 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.638+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25e40a1500 0x7f25e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.639+0000 7f25fd59b700 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b8b140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f25f40d0b50 tx=0x7f25f40d0e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7847899b6215dd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.639+0000 7f25e37fe700 1 -- 172.21.15.66:0/2255839676 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4096210 con 0x55e288b8a830 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.639+0000 7f25e37fe700 1 -- 172.21.15.66:0/2255839676 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25f4099ca0 con 0x55e288b8a830 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.639+0000 7f25e37fe700 1 -- 172.21.15.66:0/2255839676 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40042d0 con 0x55e288b8a830 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 -- 172.21.15.66:0/2255839676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 msgr2=0x55e288b8b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25e40a1500 0x7f25e40a39c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25f00076f0 tx=0x7f25f000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b8b140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f25f40d0b50 tx=0x7f25f40d0e60 comp rx=0 tx=0).stop 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 -- 172.21.15.66:0/2255839676 shutdown_connections 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b8bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 --2- 172.21.15.66:0/2255839676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b8b140 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 -- 172.21.15.66:0/2255839676 >> 172.21.15.66:0/2255839676 conn(0x55e288ab9780 msgr2=0x55e288b86880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 -- 172.21.15.66:0/2255839676 shutdown_connections 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 -- 172.21.15.66:0/2255839676 wait complete. 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.640+0000 7f26131beb80 1 -- start start 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.641+0000 7f26131beb80 1 Processor -- start 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b89ed0 0x55e288b93e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b94390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b8b720 0x55e288b9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288a98500 con 0x55e288b8a830 2023-12-11T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288a981a0 con 0x55e288b8b720 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e28895d8c0 con 0x55e288b89ed0 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f26131beb80 10 client.0 _ll_register_callbacks cb 0x55e2889c3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fcd9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b94390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fcd9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b94390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57990/0 (socket says 172.21.15.66:57990) 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fcd9a700 1 -- 172.21.15.66:0/2494835745 learned_addr learned my addr 172.21.15.66:0/2494835745 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fd59b700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b89ed0 0x55e288b93e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b8b720 0x55e288b9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fcd9a700 1 -- 172.21.15.66:0/2494835745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b89ed0 msgr2=0x55e288b93e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.642+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b89ed0 0x55e288b93e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25fcd9a700 1 -- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b8b720 msgr2=0x55e288b9b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b8b720 0x55e288b9b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25fcd9a700 1 -- 172.21.15.66:0/2494835745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f0009050 con 0x55e288b8a830 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b8b720 0x55e288b9b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b94390 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009990 tx=0x7f25e80053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4634b9234a000e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25e0ff9700 1 -- 172.21.15.66:0/2494835745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8003a90 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e288b7d730 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.643+0000 7f25e0ff9700 1 -- 172.21.15.66:0/2494835745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25e8003bf0 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f25e0ff9700 1 -- 172.21.15.66:0/2494835745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25e800b710 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 10 client.15720 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f25f81a1bd0 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f25e0ff9700 1 -- 172.21.15.66:0/2494835745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25e8032020 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f25e0ff9700 1 -- 172.21.15.66:0/2494835745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25e8004a20 con 0x55e288b8a830 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 20 client.15720 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288b9fd30 0x55e288ba2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 4 client.15720 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.644+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e288b59ee0 con 0x55e288b9fd30 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.645+0000 7f25fd59b700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288b9fd30 0x55e288ba2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.645+0000 7f25fd59b700 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288b9fd30 0x55e288ba2150 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf648b03c6360cab server_cookie=bb76d7a480b65088 in_seq=0 out_seq=0 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.645+0000 7f25e0ff9700 10 client.15720 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.646+0000 7f25e0ff9700 1 -- 172.21.15.66:0/2494835745 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e288b59ee0 con 0x55e288b9fd30 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.646+0000 7f25e0ff9700 10 client.15720 handle_command_reply: tid=0 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.646+0000 7f26131beb80 1 client.15720 shutdown 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.646+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 msgr2=0x55e288b94390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.646+0000 7f26131beb80 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b94390 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009990 tx=0x7f25e80053b0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 shutdown_connections 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288b9fd30 0x55e288ba2150 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b8b720 0x55e288b9b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b89ed0 0x55e288b93e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 --2- 172.21.15.66:0/2494835745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b94390 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 >> 172.21.15.66:0/2494835745 conn(0x55e288ab9780 msgr2=0x55e288b84ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f25e0ff9700 0 client.15720 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 shutdown_connections 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 1 -- 172.21.15.66:0/2494835745 wait complete. 2023-12-11T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.647+0000 7f26131beb80 20 client.15720 trim_cache size 0 max 16384 2023-12-11T19:34:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.664+0000 7f26131beb80 1 Processor -- start 2023-12-11T19:34:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.664+0000 7f26131beb80 1 -- start start 2023-12-11T19:34:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f26131beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f26131beb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b9b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f26131beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b93150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f26131beb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288b926b0 con 0x55e288b8a830 2023-12-11T19:34:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f26131beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288b91910 con 0x55e288b89ed0 2023-12-11T19:34:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f26131beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288b9be60 con 0x55e288b8b720 2023-12-11T19:34:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b9b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b9b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57998/0 (socket says 172.21.15.66:57998) 2023-12-11T19:34:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 -- 172.21.15.66:0/1267726018 learned_addr learned my addr 172.21.15.66:0/1267726018 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fd59b700 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 -- 172.21.15.66:0/1267726018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 msgr2=0x55e288b93150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b93150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 -- 172.21.15.66:0/1267726018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 msgr2=0x55e288b8bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fcd9a700 1 -- 172.21.15.66:0/1267726018 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e288907d80 con 0x55e288b8a830 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.665+0000 7f25fd59b700 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.666+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b9b180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25e80098e0 tx=0x7f25e8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb2c6ec657c5e48 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.666+0000 7f25e3fff700 1 -- 172.21.15.66:0/1267726018 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8003a90 con 0x55e288b8a830 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.666+0000 7f25e3fff700 1 -- 172.21.15.66:0/1267726018 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25e8032030 con 0x55e288b8a830 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.666+0000 7f25e3fff700 1 -- 172.21.15.66:0/1267726018 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e800b970 con 0x55e288b8a830 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 -- 172.21.15.66:0/1267726018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 msgr2=0x55e288b9b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b9b180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25e80098e0 tx=0x7f25e8005fb0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 -- 172.21.15.66:0/1267726018 shutdown_connections 2023-12-11T19:34:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b8bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8b720 0x55e288b93150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 --2- 172.21.15.66:0/1267726018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8a830 0x55e288b9b180 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 -- 172.21.15.66:0/1267726018 >> 172.21.15.66:0/1267726018 conn(0x55e288ab9780 msgr2=0x55e288b91c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 -- 172.21.15.66:0/1267726018 shutdown_connections 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 -- 172.21.15.66:0/1267726018 wait complete. 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.667+0000 7f26131beb80 1 -- start start 2023-12-11T19:34:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 Processor -- start 2023-12-11T19:34:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b98d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8a830 0x55e288b992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8b720 0x55e288b9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288b9a7d0 con 0x55e288b8b720 2023-12-11T19:34:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288b9a940 con 0x55e288b89ed0 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e288b9ab00 con 0x55e288b8a830 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.668+0000 7f26131beb80 10 client.0 _ll_register_callbacks cb 0x55e2889c3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fdd9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8b720 0x55e288b9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b98d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fcd9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8a830 0x55e288b992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b98d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:46264/0 (socket says 172.21.15.66:46264) 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 -- 172.21.15.66:0/994386523 learned_addr learned my addr 172.21.15.66:0/994386523 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 -- 172.21.15.66:0/994386523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8a830 msgr2=0x55e288b992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8a830 0x55e288b992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 -- 172.21.15.66:0/994386523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8b720 msgr2=0x55e288b9a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8b720 0x55e288b9a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 -- 172.21.15.66:0/994386523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f81a1bd0 con 0x55e288b89ed0 2023-12-11T19:34:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fdd9c700 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8b720 0x55e288b9a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25fd59b700 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b98d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25f4099bd0 tx=0x7f25f40ca9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f109ac7ba012d794 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.669+0000 7f25e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f25e0ff9700 1 -- 172.21.15.66:0/994386523 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4014070 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25f4099ca0 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f25e0ff9700 1 -- 172.21.15.66:0/994386523 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25f40d2070 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f25e0ff9700 1 -- 172.21.15.66:0/994386523 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25f400de60 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f26131beb80 10 client.25481 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e2888f0c00 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f25e0ff9700 1 -- 172.21.15.66:0/994386523 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25f4099ca0 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.670+0000 7f25e0ff9700 1 -- 172.21.15.66:0/994386523 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25f40965e0 con 0x55e288b89ed0 2023-12-11T19:34:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f26131beb80 10 client.25481 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f26131beb80 10 client.25481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f26131beb80 20 client.25481 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f26131beb80 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288bf6b10 0x55e288bf8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f26131beb80 4 client.25481 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e288aa4f70 con 0x55e288bf6b10 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.671+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288bf6b10 0x55e288bf8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.672+0000 7f25fcd9a700 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288bf6b10 0x55e288bf8f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abe4e019cedefaa server_cookie=67bba69e8282f7f7 in_seq=0 out_seq=0 2023-12-11T19:34:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.672+0000 7f25e0ff9700 10 client.25481 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.676+0000 7f25e0ff9700 1 -- 172.21.15.66:0/994386523 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e288aa4f70 con 0x55e288bf6b10 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.676+0000 7f25e0ff9700 10 client.25481 handle_command_reply: tid=0 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.676+0000 7f26131beb80 1 client.25481 shutdown 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.676+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 msgr2=0x55e288b98d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.676+0000 7f26131beb80 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b98d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25f4099bd0 tx=0x7f25f40ca9d0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 shutdown_connections 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e288bf6b10 0x55e288bf8f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e288b89ed0 0x55e288b98d90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e288b8b720 0x55e288b9a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 --2- 172.21.15.66:0/994386523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e288b8a830 0x55e288b992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f25e0ff9700 0 client.25481 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 >> 172.21.15.66:0/994386523 conn(0x55e288ab9780 msgr2=0x55e288b8e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 shutdown_connections 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 1 -- 172.21.15.66:0/994386523 wait complete. 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.677+0000 7f26131beb80 20 client.25481 trim_cache size 0 max 16384 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25e40a1500 msgr2=0x7f25e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25e40a1500 0x7f25e40a39c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25f00076f0 tx=0x7f25f000b040 comp rx=0 tx=0).stop 2023-12-11T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 msgr2=0x7f25f8199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8199590 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f25f400d980 tx=0x7f25f400dd40 comp rx=0 tx=0).stop 2023-12-11T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 shutdown_connections 2023-12-11T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25e40a1500 0x7f25e40a39c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8109420 0x7f25f81a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25f8073fd0 0x7f25f8199590 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 --2- 172.21.15.66:0/1824540505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25f80755b0 0x7f25f8199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 >> 172.21.15.66:0/1824540505 conn(0x7f25f80fcac0 msgr2=0x7f25f8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 shutdown_connections 2023-12-11T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:25.678+0000 7f25ff7ff700 1 -- 172.21.15.66:0/1824540505 wait complete. 2023-12-11T19:34:26.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:25 smithi167 ceph-mon[102750]: pgmap v385: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:26.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:25 smithi066 ceph-mon[97920]: pgmap v385: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:26.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:25 smithi070 ceph-mon[104044]: pgmap v385: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:28.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:27 smithi167 ceph-mon[102750]: pgmap v386: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:28.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:27 smithi066 ceph-mon[97920]: pgmap v386: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:28.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:27 smithi070 ceph-mon[104044]: pgmap v386: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:29 smithi167 ceph-mon[102750]: pgmap v387: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-12-11T19:34:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:29 smithi066 ceph-mon[97920]: pgmap v387: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-12-11T19:34:30.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:29 smithi070 ceph-mon[104044]: pgmap v387: 97 pgs: 97 active+clean; 662 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-12-11T19:34:32.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:31 smithi167 ceph-mon[102750]: pgmap v388: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-11T19:34:32.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:31 smithi066 ceph-mon[97920]: pgmap v388: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-11T19:34:32.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:31 smithi070 ceph-mon[104044]: pgmap v388: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-11T19:34:34.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:33 smithi167 ceph-mon[102750]: pgmap v389: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:34.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:33 smithi066 ceph-mon[97920]: pgmap v389: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:34.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:33 smithi070 ceph-mon[104044]: pgmap v389: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:36.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:35 smithi167 ceph-mon[102750]: pgmap v390: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:36.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:35 smithi066 ceph-mon[97920]: pgmap v390: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:36.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:35 smithi070 ceph-mon[104044]: pgmap v390: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:38.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:37 smithi167 ceph-mon[102750]: pgmap v391: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:38.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:37 smithi066 ceph-mon[97920]: pgmap v391: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:38.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:37 smithi070 ceph-mon[104044]: pgmap v391: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 -- 172.21.15.66:0/1117614644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 msgr2=0x7f809c104390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 --2- 172.21.15.66:0/1117614644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 0x7f809c104390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8098009990 tx=0x7f8098009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 -- 172.21.15.66:0/1117614644 shutdown_connections 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 --2- 172.21.15.66:0/1117614644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 --2- 172.21.15.66:0/1117614644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 0x7f809c104390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 --2- 172.21.15.66:0/1117614644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 0x7f809c106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.923+0000 7f80a47ff700 1 -- 172.21.15.66:0/1117614644 >> 172.21.15.66:0/1117614644 conn(0x7f809c0fb990 msgr2=0x7f809c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.924+0000 7f80a47ff700 1 -- 172.21.15.66:0/1117614644 shutdown_connections 2023-12-11T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.924+0000 7f80a47ff700 1 -- 172.21.15.66:0/1117614644 wait complete. 2023-12-11T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.925+0000 7f80a47ff700 1 Processor -- start 2023-12-11T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.925+0000 7f80a47ff700 1 -- start start 2023-12-11T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a47ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 0x7f809c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a47ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 0x7f809c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a47ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a47ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1a10f0 con 0x7f809c1048d0 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a47ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1a1230 con 0x7f809c107200 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a47ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c1a13c0 con 0x7f809c101fa0 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a1d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 0x7f809c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:59308/0 (socket says 172.21.15.66:59308) 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.926+0000 7f80a2d9c700 1 -- 172.21.15.66:0/4037849310 learned_addr learned my addr 172.21.15.66:0/4037849310 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a259b700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 0x7f809c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a1d9a700 1 -- 172.21.15.66:0/4037849310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 msgr2=0x7f809c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a1d9a700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 0x7f809c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a1d9a700 1 -- 172.21.15.66:0/4037849310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 msgr2=0x7f809c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a1d9a700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a1d9a700 1 -- 172.21.15.66:0/4037849310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980095f0 con 0x7f809c1048d0 2023-12-11T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.927+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.928+0000 7f80a1d9a700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 0x7f809c199b00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f808c00d750 tx=0x7f808c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.928+0000 7f80937fe700 1 -- 172.21.15.66:0/4037849310 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c004020 con 0x7f809c1048d0 2023-12-11T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.928+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c1a16a0 con 0x7f809c1048d0 2023-12-11T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.928+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c1a1c00 con 0x7f809c1048d0 2023-12-11T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.928+0000 7f80937fe700 1 -- 172.21.15.66:0/4037849310 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f808c004830 con 0x7f809c1048d0 2023-12-11T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.928+0000 7f80937fe700 1 -- 172.21.15.66:0/4037849310 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c005390 con 0x7f809c1048d0 2023-12-11T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.934+0000 7f80937fe700 1 -- 172.21.15.66:0/4037849310 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f808c0054f0 con 0x7f809c1048d0 2023-12-11T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.935+0000 7f80b81d8b80 1 Processor -- start 2023-12-11T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.936+0000 7f80b81d8b80 1 -- start start 2023-12-11T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.936+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb4c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80937fe700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f80880a1500 0x7f80880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb4cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accaa6af0 con 0x560accb4aed0 2023-12-11T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accaa4410 con 0x560accb4c720 2023-12-11T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acca9fd00 con 0x560accb4b830 2023-12-11T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80937fe700 1 -- 172.21.15.66:0/4037849310 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f808c0cc5a0 con 0x7f809c1048d0 2023-12-11T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80a259b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb4c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.937+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb4cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb4cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:59310/0 (socket says 172.21.15.66:59310) 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 -- 172.21.15.66:0/2331759751 learned_addr learned my addr 172.21.15.66:0/2331759751 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a1d9a700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f80880a1500 0x7f80880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 -- 172.21.15.66:0/2331759751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 msgr2=0x560accb4c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb4c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 -- 172.21.15.66:0/2331759751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 msgr2=0x560accb4cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb4cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.938+0000 7f80a2d9c700 1 -- 172.21.15.66:0/2331759751 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560acc91c5d0 con 0x560accb4aed0 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.939+0000 7f80a1d9a700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f80880a1500 0x7f80880a39c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f808c0a16e0 tx=0x7f808c004750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.939+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb4b2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8094009550 tx=0x7f8094009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e6d8eeadbf113b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.939+0000 7f8090ff9700 1 -- 172.21.15.66:0/2331759751 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094004db0 con 0x560accb4aed0 2023-12-11T19:34:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.939+0000 7f8090ff9700 1 -- 172.21.15.66:0/2331759751 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f809400f040 con 0x560accb4aed0 2023-12-11T19:34:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.939+0000 7f8090ff9700 1 -- 172.21.15.66:0/2331759751 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094004290 con 0x560accb4aed0 2023-12-11T19:34:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.940+0000 7f80b81d8b80 1 -- 172.21.15.66:0/2331759751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 msgr2=0x560accb4b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.940+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb4b2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8094009550 tx=0x7f8094009920 comp rx=0 tx=0).stop 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 -- 172.21.15.66:0/2331759751 shutdown_connections 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb4cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb4b2f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/2331759751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb4c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 -- 172.21.15.66:0/2331759751 >> 172.21.15.66:0/2331759751 conn(0x560acca7a780 msgr2=0x560accb47880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 -- 172.21.15.66:0/2331759751 shutdown_connections 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.941+0000 7f80b81d8b80 1 -- 172.21.15.66:0/2331759751 wait complete. 2023-12-11T19:34:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.942+0000 7f80b81d8b80 1 -- start start 2023-12-11T19:34:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.942+0000 7f80b81d8b80 1 Processor -- start 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4b830 0x560accb55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4c720 0x560accb5c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acca59500 con 0x560accb4aed0 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acca591a0 con 0x560accb4b830 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560acc91e8c0 con 0x560accb4c720 2023-12-11T19:34:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80b81d8b80 10 client.0 _ll_register_callbacks cb 0x560acc984a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.943+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb54ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39902/0 (socket says 172.21.15.66:39902) 2023-12-11T19:34:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 -- 172.21.15.66:0/767276466 learned_addr learned my addr 172.21.15.66:0/767276466 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a259b700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4b830 0x560accb55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 -- 172.21.15.66:0/767276466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4c720 msgr2=0x560accb5c260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4c720 0x560accb5c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 -- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4b830 msgr2=0x560accb55430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4b830 0x560accb55430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a2d9c700 1 -- 172.21.15.66:0/767276466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8094007730 con 0x560accb4aed0 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.944+0000 7f80a259b700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4b830 0x560accb55430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.945+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb54ef0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f809400c040 tx=0x7f8094004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b35055bbf136738 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.945+0000 7f8085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.945+0000 7f8085ffb700 1 -- 172.21.15.66:0/767276466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094003c00 con 0x560accb4aed0 2023-12-11T19:34:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.945+0000 7f8085ffb700 1 -- 172.21.15.66:0/767276466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f809400ee70 con 0x560accb4aed0 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.945+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560accb3e730 con 0x560accb4aed0 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.946+0000 7f8085ffb700 1 -- 172.21.15.66:0/767276466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8094003d60 con 0x560accb4aed0 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.946+0000 7f80b81d8b80 10 client.15738 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.946+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560acc8c8d80 con 0x560accb4aed0 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.946+0000 7f8085ffb700 1 -- 172.21.15.66:0/767276466 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80940044d0 con 0x560accb4aed0 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f8085ffb700 1 -- 172.21.15.66:0/767276466 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f809401a650 con 0x560accb4aed0 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80b81d8b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80b81d8b80 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80b81d8b80 20 client.15738 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accb60c70 0x560accb630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80b81d8b80 4 client.15738 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560accb3e730 con 0x560accb60c70 2023-12-11T19:34:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.947+0000 7f80a259b700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accb60c70 0x560accb630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.948+0000 7f80a259b700 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accb60c70 0x560accb630b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9525fbde0cdfccb4 server_cookie=4d7d01cbde0f3f0d in_seq=0 out_seq=0 2023-12-11T19:34:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.948+0000 7f8085ffb700 10 client.15738 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.949+0000 7f8085ffb700 1 -- 172.21.15.66:0/767276466 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560accb3e730 con 0x560accb60c70 2023-12-11T19:34:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.949+0000 7f8085ffb700 10 client.15738 handle_command_reply: tid=0 2023-12-11T19:34:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.949+0000 7f80b81d8b80 1 client.15738 shutdown 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 msgr2=0x560accb54ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb54ef0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f809400c040 tx=0x7f8094004d00 comp rx=0 tx=0).stop 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 shutdown_connections 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accb60c70 0x560accb630b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4c720 0x560accb5c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb54ef0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/767276466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4b830 0x560accb55430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f8085ffb700 0 client.15738 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 >> 172.21.15.66:0/767276466 conn(0x560acca7a780 msgr2=0x560accb45ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 shutdown_connections 2023-12-11T19:34:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 1 -- 172.21.15.66:0/767276466 wait complete. 2023-12-11T19:34:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.950+0000 7f80b81d8b80 20 client.15738 trim_cache size 0 max 16384 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.981+0000 7f80b81d8b80 1 Processor -- start 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 -- start start 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4aed0 0x560accb541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 0x560accb5c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb47460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accb60a10 con 0x560accb4b830 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accb682a0 con 0x560accb4c720 2023-12-11T19:34:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accb50170 con 0x560accb4aed0 2023-12-11T19:34:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.982+0000 7f80a259b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 0x560accb5c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 0x560accb5c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39908/0 (socket says 172.21.15.66:39908) 2023-12-11T19:34:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 -- 172.21.15.66:0/1559335016 learned_addr learned my addr 172.21.15.66:0/1559335016 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 -- 172.21.15.66:0/1559335016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4aed0 msgr2=0x560accb541f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4aed0 0x560accb541f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 -- 172.21.15.66:0/1559335016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 msgr2=0x560accb47460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb47460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.983+0000 7f80a259b700 1 -- 172.21.15.66:0/1559335016 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560acc8b1c00 con 0x560accb4b830 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.984+0000 7f80a259b700 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 0x560accb5c1f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8098009990 tx=0x7f809800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8882be0e5d0e719d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.984+0000 7f80917fa700 1 -- 172.21.15.66:0/1559335016 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809802a070 con 0x560accb4b830 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.984+0000 7f80917fa700 1 -- 172.21.15.66:0/1559335016 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f809801e040 con 0x560accb4b830 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.984+0000 7f80917fa700 1 -- 172.21.15.66:0/1559335016 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809801d040 con 0x560accb4b830 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 -- 172.21.15.66:0/1559335016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 msgr2=0x560accb5c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 0x560accb5c1f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8098009990 tx=0x7f809800b610 comp rx=0 tx=0).stop 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 -- 172.21.15.66:0/1559335016 shutdown_connections 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb47460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4aed0 0x560accb541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/1559335016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4b830 0x560accb5c1f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 -- 172.21.15.66:0/1559335016 >> 172.21.15.66:0/1559335016 conn(0x560acca7a780 msgr2=0x560accb53a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 -- 172.21.15.66:0/1559335016 shutdown_connections 2023-12-11T19:34:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.985+0000 7f80b81d8b80 1 -- 172.21.15.66:0/1559335016 wait complete. 2023-12-11T19:34:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.986+0000 7f80b81d8b80 1 -- start start 2023-12-11T19:34:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.986+0000 7f80b81d8b80 1 Processor -- start 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb5a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb5a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb5b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accb5bcf0 con 0x560accb4aed0 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accb5be60 con 0x560accb4c720 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560accb5c020 con 0x560accb4b830 2023-12-11T19:34:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80b81d8b80 10 client.0 _ll_register_callbacks cb 0x560acc984a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.987+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb5a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb5a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39920/0 (socket says 172.21.15.66:39920) 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 -- 172.21.15.66:0/3475968724 learned_addr learned my addr 172.21.15.66:0/3475968724 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb5b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 -- 172.21.15.66:0/3475968724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 msgr2=0x560accb5a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a259b700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb5a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb5a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 -- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 msgr2=0x560accb5b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb5b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.988+0000 7f80a2d9c700 1 -- 172.21.15.66:0/3475968724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c10a260 con 0x560accb4aed0 2023-12-11T19:34:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.989+0000 7f80a2d9c700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb5a380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8094004ed0 tx=0x7f8094009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43b42786c27a665 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.989+0000 7f8085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.989+0000 7f8085ffb700 1 -- 172.21.15.66:0/3475968724 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400eae0 con 0x560accb4aed0 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.989+0000 7f8085ffb700 1 -- 172.21.15.66:0/3475968724 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f809400ec40 con 0x560accb4aed0 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.989+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f809400f040 con 0x560accb4aed0 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f8085ffb700 1 -- 172.21.15.66:0/3475968724 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f809400ee00 con 0x560accb4aed0 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 10 client.15744 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560acc8ef1f0 con 0x560accb4aed0 2023-12-11T19:34:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f8085ffb700 1 -- 172.21.15.66:0/3475968724 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f809400f040 con 0x560accb4aed0 2023-12-11T19:34:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f8085ffb700 1 -- 172.21.15.66:0/3475968724 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8094004370 con 0x560accb4aed0 2023-12-11T19:34:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 20 client.15744 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accbc08c0 0x560accbc2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 4 client.15744 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:34:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.990+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560accb1aee0 con 0x560accbc08c0 2023-12-11T19:34:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.991+0000 7f80a259b700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accbc08c0 0x560accbc2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.992+0000 7f8085ffb700 10 client.15744 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.992+0000 7f80a259b700 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accbc08c0 0x560accbc2ce0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42f509f2bf26728 server_cookie=f0fbe24d8da0dc3a in_seq=0 out_seq=0 2023-12-11T19:34:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.996+0000 7f8085ffb700 1 -- 172.21.15.66:0/3475968724 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560accb1aee0 con 0x560accbc08c0 2023-12-11T19:34:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.996+0000 7f8085ffb700 10 client.15744 handle_command_reply: tid=0 2023-12-11T19:34:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.996+0000 7f80b81d8b80 1 client.15744 shutdown 2023-12-11T19:34:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 msgr2=0x560accb5a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb5a380 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8094004ed0 tx=0x7f8094009a50 comp rx=0 tx=0).stop 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 shutdown_connections 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x560accbc08c0 0x560accbc2ce0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560accb4c720 0x560accb5b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560accb4aed0 0x560accb5a380 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 --2- 172.21.15.66:0/3475968724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560accb4b830 0x560accb5a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 >> 172.21.15.66:0/3475968724 conn(0x560acca7a780 msgr2=0x560accb47460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.997+0000 7f8085ffb700 0 client.15744 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.998+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 shutdown_connections 2023-12-11T19:34:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.998+0000 7f80b81d8b80 1 -- 172.21.15.66:0/3475968724 wait complete. 2023-12-11T19:34:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.998+0000 7f80b81d8b80 20 client.15744 trim_cache size 0 max 16384 2023-12-11T19:34:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:34:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:34:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:34:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.999+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f80880a1500 msgr2=0x7f80880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:38.999+0000 7f80a47ff700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f80880a1500 0x7f80880a39c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f808c0a16e0 tx=0x7f808c004750 comp rx=0 tx=0).stop 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 msgr2=0x7f809c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 0x7f809c199b00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f808c00d750 tx=0x7f808c00da60 comp rx=0 tx=0).stop 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 shutdown_connections 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f80880a1500 0x7f80880a39c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f809c107200 0x7f809c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f809c101fa0 0x7f809c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 --2- 172.21.15.66:0/4037849310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c1048d0 0x7f809c199b00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 >> 172.21.15.66:0/4037849310 conn(0x7f809c0fb990 msgr2=0x7f809c100030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 shutdown_connections 2023-12-11T19:34:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:39.000+0000 7f80a47ff700 1 -- 172.21.15.66:0/4037849310 wait complete. 2023-12-11T19:34:40.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:39 smithi167 ceph-mon[102750]: pgmap v392: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:40.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:39 smithi066 ceph-mon[97920]: pgmap v392: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:40.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:39 smithi070 ceph-mon[104044]: pgmap v392: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:42.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:41 smithi167 ceph-mon[102750]: pgmap v393: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-11T19:34:42.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:41 smithi066 ceph-mon[97920]: pgmap v393: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-11T19:34:42.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:41 smithi070 ceph-mon[104044]: pgmap v393: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-11T19:34:44.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:43 smithi167 ceph-mon[102750]: pgmap v394: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:44.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:43 smithi066 ceph-mon[97920]: pgmap v394: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:44.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:43 smithi070 ceph-mon[104044]: pgmap v394: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:46.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:45 smithi167 ceph-mon[102750]: pgmap v395: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:46.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:45 smithi066 ceph-mon[97920]: pgmap v395: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:46.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:45 smithi070 ceph-mon[104044]: pgmap v395: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:48.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:47 smithi167 ceph-mon[102750]: pgmap v396: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:48.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:47 smithi066 ceph-mon[97920]: pgmap v396: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:48.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:47 smithi070 ceph-mon[104044]: pgmap v396: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-11T19:34:50.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:49 smithi167 ceph-mon[102750]: pgmap v397: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:50.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:49 smithi066 ceph-mon[97920]: pgmap v397: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:50.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:49 smithi070 ceph-mon[104044]: pgmap v397: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-11T19:34:50.458 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:34:50.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/1005465754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34102000 msgr2=0x7f2c341043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/1005465754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34102000 0x7f2c341043f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009990 tx=0x7f2c30009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/1005465754 shutdown_connections 2023-12-11T19:34:50.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/1005465754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34107260 0x7f2c341096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/1005465754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34102000 0x7f2c341043f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/1005465754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34104930 0x7f2c34106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/1005465754 >> 172.21.15.66:0/1005465754 conn(0x7f2c340fb9f0 msgr2=0x7f2c340fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/1005465754 shutdown_connections 2023-12-11T19:34:50.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.812+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/1005465754 wait complete. 2023-12-11T19:34:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.814+0000 7f2c3c1ff700 1 Processor -- start 2023-12-11T19:34:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.814+0000 7f2c3c1ff700 1 -- start start 2023-12-11T19:34:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3c1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 0x7f2c34195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3c1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34104930 0x7f2c34195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3c1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34107260 0x7f2c341a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3c1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341a5620 con 0x7f2c34104930 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3c1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341a5760 con 0x7f2c34107260 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3c1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341a58a0 con 0x7f2c34102000 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c39f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 0x7f2c34195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3979a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34104930 0x7f2c34195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c39f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 0x7f2c34195210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:57222/0 (socket says 172.21.15.66:57222) 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c39f9b700 1 -- 172.21.15.66:0/3022127005 learned_addr learned my addr 172.21.15.66:0/3022127005 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.815+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34107260 0x7f2c341a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c39f9b700 1 -- 172.21.15.66:0/3022127005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34107260 msgr2=0x7f2c341a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c39f9b700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34107260 0x7f2c341a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c39f9b700 1 -- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34104930 msgr2=0x7f2c34195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c39f9b700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34104930 0x7f2c34195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c39f9b700 1 -- 172.21.15.66:0/3022127005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300095f0 con 0x7f2c34102000 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34107260 0x7f2c341a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.816+0000 7f2c3979a700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34104930 0x7f2c34195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.817+0000 7f2c39f9b700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 0x7f2c34195210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c30003ec0 tx=0x7f2c30004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.817+0000 7f2c2affd700 1 -- 172.21.15.66:0/3022127005 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3001d070 con 0x7f2c34102000 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.817+0000 7f2c2affd700 1 -- 172.21.15.66:0/3022127005 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c3000bdb0 con 0x7f2c34102000 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.817+0000 7f2c2affd700 1 -- 172.21.15.66:0/3022127005 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000fda0 con 0x7f2c34102000 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.817+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c341a5aa0 con 0x7f2c34102000 2023-12-11T19:34:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.817+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c341a5e70 con 0x7f2c34102000 2023-12-11T19:34:50.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.823+0000 7f2c2affd700 1 -- 172.21.15.66:0/3022127005 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2c3000f460 con 0x7f2c34102000 2023-12-11T19:34:50.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.824+0000 7f2c4fa74b80 1 Processor -- start 2023-12-11T19:34:50.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.824+0000 7f2c4fa74b80 1 -- start start 2023-12-11T19:34:50.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b1e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b1f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70a79b50 con 0x561f70b1e3d0 2023-12-11T19:34:50.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70a77470 con 0x561f70b1fc20 2023-12-11T19:34:50.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70a72d60 con 0x561f70b1ed30 2023-12-11T19:34:50.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b1e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b1e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55896/0 (socket says 172.21.15.66:55896) 2023-12-11T19:34:50.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.825+0000 7f2c3a79c700 1 -- 172.21.15.66:0/3353197997 learned_addr learned my addr 172.21.15.66:0/3353197997 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.826+0000 7f2c3a79c700 1 -- 172.21.15.66:0/3353197997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 msgr2=0x561f70b1f640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.826+0000 7f2c3979a700 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b1f640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c3a79c700 1 -- 172.21.15.66:0/3353197997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 msgr2=0x561f70b200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c3a79c700 1 -- 172.21.15.66:0/3353197997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f708ef630 con 0x561f70b1e3d0 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c3979a700 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c2affd700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c200a1580 0x7f2c200a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.827+0000 7f2c2affd700 1 -- 172.21.15.66:0/3022127005 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c300cdf80 con 0x7f2c34102000 2023-12-11T19:34:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.828+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b1e7f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009440 tx=0x7f2c2c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4e6e641d2b6e51 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.828+0000 7f2c39f9b700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c200a1580 0x7f2c200a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.828+0000 7f2c1ffff700 1 -- 172.21.15.66:0/3353197997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00e070 con 0x561f70b1e3d0 2023-12-11T19:34:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.828+0000 7f2c1ffff700 1 -- 172.21.15.66:0/3353197997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c2c00f040 con 0x561f70b1e3d0 2023-12-11T19:34:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.828+0000 7f2c1ffff700 1 -- 172.21.15.66:0/3353197997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c014a10 con 0x561f70b1e3d0 2023-12-11T19:34:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/3353197997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 msgr2=0x561f70b1e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c39f9b700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c200a1580 0x7f2c200a3a40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c300977f0 tx=0x7f2c30009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b1e7f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009440 tx=0x7f2c2c009920 comp rx=0 tx=0).stop 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/3353197997 shutdown_connections 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b1e7f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/3353197997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b1f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/3353197997 >> 172.21.15.66:0/3353197997 conn(0x561f70a4d7e0 msgr2=0x561f70b1ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/3353197997 shutdown_connections 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.829+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/3353197997 wait complete. 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.830+0000 7f2c4fa74b80 1 -- start start 2023-12-11T19:34:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.830+0000 7f2c4fa74b80 1 Processor -- start 2023-12-11T19:34:50.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b28710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b2f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70a2c560 con 0x561f70b1e3d0 2023-12-11T19:34:50.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70a2c200 con 0x561f70b1fc20 2023-12-11T19:34:50.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f708f1920 con 0x561f70b1ed30 2023-12-11T19:34:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.831+0000 7f2c4fa74b80 10 client.0 _ll_register_callbacks cb 0x561f70957a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55906/0 (socket says 172.21.15.66:55906) 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1326474472 learned_addr learned my addr 172.21.15.66:0/1326474472 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b28710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b2f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1326474472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 msgr2=0x561f70b28710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b28710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 msgr2=0x561f70b2f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.832+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b2f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.833+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1326474472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c0090f0 con 0x561f70b1e3d0 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.833+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b281d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0144e0 tx=0x7f2c2c009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612cdb798b48498b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.833+0000 7f2c1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.833+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1326474472 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00e040 con 0x561f70b1e3d0 2023-12-11T19:34:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.833+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1326474472 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c2c003c00 con 0x561f70b1e3d0 2023-12-11T19:34:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.833+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f70b11b40 con 0x561f70b1e3d0 2023-12-11T19:34:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.834+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1326474472 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c003dc0 con 0x561f70b1e3d0 2023-12-11T19:34:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.834+0000 7f2c4fa74b80 10 client.15756 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.834+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f7089bde0 con 0x561f70b1e3d0 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.834+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1326474472 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f70b11b40 con 0x561f70b1e3d0 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1326474472 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c2c01c070 con 0x561f70b1e3d0 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c4fa74b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c4fa74b80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c4fa74b80 20 client.15756 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b33df0 0x561f70b36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c4fa74b80 4 client.15756 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.835+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f70aedf40 con 0x561f70b33df0 2023-12-11T19:34:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.836+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b33df0 0x561f70b36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.836+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b33df0 0x561f70b36210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a19d4074af72f224 server_cookie=e0fe539c2e50b804 in_seq=0 out_seq=0 2023-12-11T19:34:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.836+0000 7f2c1d7fa700 10 client.15756 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:50.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.837+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1326474472 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f70aedf40 con 0x561f70b33df0 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.837+0000 7f2c1d7fa700 10 client.15756 handle_command_reply: tid=0 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.837+0000 7f2c4fa74b80 1 client.15756 shutdown 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 msgr2=0x561f70b281d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b281d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0144e0 tx=0x7f2c2c009600 comp rx=0 tx=0).stop 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 shutdown_connections 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b33df0 0x561f70b36210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1fc20 0x561f70b2f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1e3d0 0x561f70b281d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1326474472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1ed30 0x561f70b28710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 >> 172.21.15.66:0/1326474472 conn(0x561f70a4d7e0 msgr2=0x561f70b19a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c1d7fa700 0 client.15756 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 shutdown_connections 2023-12-11T19:34:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.838+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1326474472 wait complete. 2023-12-11T19:34:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.839+0000 7f2c4fa74b80 20 client.15756 trim_cache size 0 max 16384 2023-12-11T19:34:50.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 Processor -- start 2023-12-11T19:34:50.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 -- start start 2023-12-11T19:34:50.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1e3d0 0x561f70b22b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 0x561f70b230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1fc20 0x561f70b2f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70b15790 con 0x561f70b1fc20 2023-12-11T19:34:50.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70b1f110 con 0x561f70b1ed30 2023-12-11T19:34:50.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.874+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70b27bc0 con 0x561f70b1e3d0 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 0x561f70b230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1fc20 0x561f70b2f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 0x561f70b230d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36244/0 (socket says 172.21.15.66:36244) 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 -- 172.21.15.66:0/1046657849 learned_addr learned my addr 172.21.15.66:0/1046657849 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1e3d0 0x561f70b22b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 -- 172.21.15.66:0/1046657849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1e3d0 msgr2=0x561f70b22b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1e3d0 0x561f70b22b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 -- 172.21.15.66:0/1046657849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1fc20 msgr2=0x561f70b2f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1fc20 0x561f70b2f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3979a700 1 -- 172.21.15.66:0/1046657849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f70884c60 con 0x561f70b1ed30 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.875+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1fc20 0x561f70b2f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.876+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 0x561f70b230d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c24004590 tx=0x7f2c24003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff708d266f01d0a server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.876+0000 7f2c28ff9700 1 -- 172.21.15.66:0/1046657849 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24017040 con 0x561f70b1ed30 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.876+0000 7f2c28ff9700 1 -- 172.21.15.66:0/1046657849 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c24020070 con 0x561f70b1ed30 2023-12-11T19:34:50.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.876+0000 7f2c28ff9700 1 -- 172.21.15.66:0/1046657849 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24005020 con 0x561f70b1ed30 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1046657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 msgr2=0x561f70b230d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 0x561f70b230d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c24004590 tx=0x7f2c24003960 comp rx=0 tx=0).stop 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1046657849 shutdown_connections 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1fc20 0x561f70b2f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1e3d0 0x561f70b22b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1046657849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1ed30 0x561f70b230d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.877+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1046657849 >> 172.21.15.66:0/1046657849 conn(0x561f70a4d7e0 msgr2=0x561f70b26eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.878+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1046657849 shutdown_connections 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.878+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1046657849 wait complete. 2023-12-11T19:34:50.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.878+0000 7f2c4fa74b80 1 -- start start 2023-12-11T19:34:50.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.878+0000 7f2c4fa74b80 1 Processor -- start 2023-12-11T19:34:50.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 0x561f70b2d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1ed30 0x561f70b2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1fc20 0x561f70b2e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70b2ed90 con 0x561f70b1ed30 2023-12-11T19:34:50.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70b2ef00 con 0x561f70b1e3d0 2023-12-11T19:34:50.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f70b2f0c0 con 0x561f70b1fc20 2023-12-11T19:34:50.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.880+0000 7f2c4fa74b80 10 client.0 _ll_register_callbacks cb 0x561f70957a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:50.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.881+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 0x561f70b2d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.881+0000 7f2c3a79c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 0x561f70b2d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36254/0 (socket says 172.21.15.66:36254) 2023-12-11T19:34:50.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.881+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1429262505 learned_addr learned my addr 172.21.15.66:0/1429262505 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:50.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.881+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1fc20 0x561f70b2e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.881+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1ed30 0x561f70b2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1429262505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1fc20 msgr2=0x561f70b2e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1fc20 0x561f70b2e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1429262505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1ed30 msgr2=0x561f70b2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:50.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1ed30 0x561f70b2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3a79c700 1 -- 172.21.15.66:0/1429262505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f7089bde0 con 0x561f70b1e3d0 2023-12-11T19:34:50.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1ed30 0x561f70b2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:50.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c3a79c700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 0x561f70b2d350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009440 tx=0x7f2c2c017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e04533383db067 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:50.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.882+0000 7f2c1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:34:50.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.883+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1429262505 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00e040 con 0x561f70b1e3d0 2023-12-11T19:34:50.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.883+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1429262505 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c2c009990 con 0x561f70b1e3d0 2023-12-11T19:34:50.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.883+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c2c00f040 con 0x561f70b1e3d0 2023-12-11T19:34:50.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.884+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1429262505 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c003c10 con 0x561f70b1e3d0 2023-12-11T19:34:50.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.884+0000 7f2c4fa74b80 10 client.25511 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:50.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.884+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f2c24004240 con 0x561f70b1e3d0 2023-12-11T19:34:50.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.885+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1429262505 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c2c00f040 con 0x561f70b1e3d0 2023-12-11T19:34:50.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.885+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1429262505 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c2c01c070 con 0x561f70b1e3d0 2023-12-11T19:34:50.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.885+0000 7f2c4fa74b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:50.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.885+0000 7f2c4fa74b80 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.885+0000 7f2c4fa74b80 20 client.25511 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.886+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b8ac90 0x561f70b8d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.886+0000 7f2c4fa74b80 4 client.25511 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.886+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f70a38fd0 con 0x561f70b8ac90 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.886+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b8ac90 0x561f70b8d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c3979a700 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b8ac90 0x561f70b8d0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ac6039a713ef1f server_cookie=5a39800c01268a9d in_seq=0 out_seq=0 2023-12-11T19:34:50.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c1d7fa700 10 client.25511 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:50.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c1d7fa700 1 -- 172.21.15.66:0/1429262505 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561f70a38fd0 con 0x561f70b8ac90 2023-12-11T19:34:50.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c1d7fa700 10 client.25511 handle_command_reply: tid=0 2023-12-11T19:34:50.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c4fa74b80 1 client.25511 shutdown 2023-12-11T19:34:50.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 msgr2=0x561f70b2d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 0x561f70b2d350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009440 tx=0x7f2c2c017040 comp rx=0 tx=0).stop 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 shutdown_connections 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.887+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561f70b8ac90 0x561f70b8d0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561f70b1e3d0 0x561f70b2d350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f70b1fc20 0x561f70b2e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 1 --2- 172.21.15.66:0/1429262505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f70b1ed30 0x561f70b2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 >> 172.21.15.66:0/1429262505 conn(0x561f70a4d7e0 msgr2=0x561f70b22b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:50.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c1d7fa700 0 client.25511 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:50.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 shutdown_connections 2023-12-11T19:34:50.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 1 -- 172.21.15.66:0/1429262505 wait complete. 2023-12-11T19:34:50.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.888+0000 7f2c4fa74b80 20 client.25511 trim_cache size 0 max 16384 2023-12-11T19:34:50.891 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:34:50.891 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-12-11T19:34:50.892 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-12-11T19:34:50.892 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:34:50.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c200a1580 msgr2=0x7f2c200a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c200a1580 0x7f2c200a3a40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c300977f0 tx=0x7f2c30009310 comp rx=0 tx=0).stop 2023-12-11T19:34:50.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 msgr2=0x7f2c34195210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 0x7f2c34195210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c30003ec0 tx=0x7f2c30004730 comp rx=0 tx=0).stop 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 shutdown_connections 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2c200a1580 0x7f2c200a3a40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c34107260 0x7f2c341a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c34102000 0x7f2c34195210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 --2- 172.21.15.66:0/3022127005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c34104930 0x7f2c34195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.889+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 >> 172.21.15.66:0/3022127005 conn(0x7f2c340fb9f0 msgr2=0x7f2c340ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.890+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 shutdown_connections 2023-12-11T19:34:50.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:50.890+0000 7f2c3c1ff700 1 -- 172.21.15.66:0/3022127005 wait complete. 2023-12-11T19:34:50.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-11T19:34:50.918 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:34:51.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 -- 172.21.15.66:0/2403564765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 msgr2=0x7fa1ac106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2403564765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac106c10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa198009990 tx=0x7fa198009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 -- 172.21.15.66:0/2403564765 shutdown_connections 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2403564765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac107150 0x7fa1ac1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2403564765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac101ef0 0x7fa1ac1042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2403564765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac106c10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 -- 172.21.15.66:0/2403564765 >> 172.21.15.66:0/2403564765 conn(0x7fa1ac0fb900 msgr2=0x7fa1ac0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.263+0000 7fa1b199d700 1 -- 172.21.15.66:0/2403564765 shutdown_connections 2023-12-11T19:34:51.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.264+0000 7fa1b199d700 1 -- 172.21.15.66:0/2403564765 wait complete. 2023-12-11T19:34:51.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.265+0000 7fa1b199d700 1 Processor -- start 2023-12-11T19:34:51.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.265+0000 7fa1b199d700 1 -- start start 2023-12-11T19:34:51.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b199d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac101ef0 0x7fa1ac1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:51.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b199d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac0ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:51.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b199d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac107150 0x7fa1ac1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:51.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b199d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac100ae0 con 0x7fa1ac101ef0 2023-12-11T19:34:51.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b199d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac100c50 con 0x7fa1ac104820 2023-12-11T19:34:51.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b199d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac1a13f0 con 0x7fa1ac107150 2023-12-11T19:34:51.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac0ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:51.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac0ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36278/0 (socket says 172.21.15.66:36278) 2023-12-11T19:34:51.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1abfff700 1 -- 172.21.15.66:0/2859456948 learned_addr learned my addr 172.21.15.66:0/2859456948 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:51.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.266+0000 7fa1b119c700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac107150 0x7fa1ac1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:51.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b099b700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac101ef0 0x7fa1ac1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b119c700 1 -- 172.21.15.66:0/2859456948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 msgr2=0x7fa1ac0ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b119c700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac0ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b119c700 1 -- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac101ef0 msgr2=0x7fa1ac1017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b119c700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac101ef0 0x7fa1ac1017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b119c700 1 -- 172.21.15.66:0/2859456948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980095f0 con 0x7fa1ac107150 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1abfff700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac0ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b099b700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac101ef0 0x7fa1ac1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.267+0000 7fa1b119c700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac107150 0x7fa1ac1003e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4007f10 tx=0x7fa1a400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.268+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4009ea0 con 0x7fa1ac107150 2023-12-11T19:34:51.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.268+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac1a1650 con 0x7fa1ac107150 2023-12-11T19:34:51.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.268+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1a400f040 con 0x7fa1ac107150 2023-12-11T19:34:51.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.268+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4004280 con 0x7fa1ac107150 2023-12-11T19:34:51.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.268+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac1a1bb0 con 0x7fa1ac107150 2023-12-11T19:34:51.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.269+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac0fd480 con 0x7fa1ac107150 2023-12-11T19:34:51.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.273+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa1a40044a0 con 0x7fa1ac107150 2023-12-11T19:34:51.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.276+0000 7fa1a9ffb700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa19c0a1500 0x7fa19c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:51.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.276+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa1a40cc830 con 0x7fa1ac107150 2023-12-11T19:34:51.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.276+0000 7fa1b099b700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa19c0a1500 0x7fa19c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:51.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.276+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa1a4095e50 con 0x7fa1ac107150 2023-12-11T19:34:51.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.277+0000 7fa1b099b700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa19c0a1500 0x7fa19c0a39c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009cb0 tx=0x7fa1a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:51.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.792+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1ac068610 con 0x7fa1ac107150 2023-12-11T19:34:51.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.793+0000 7fa1a9ffb700 1 -- 172.21.15.66:0/2859456948 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11362 (secure 0 0 0) 0x7fa1a4019070 con 0x7fa1ac107150 2023-12-11T19:34:51.795 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:34:51.795 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:31:59.541695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa19c0a1500 msgr2=0x7fa19c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa19c0a1500 0x7fa19c0a39c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009cb0 tx=0x7fa1a0009260 comp rx=0 tx=0).stop 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac107150 msgr2=0x7fa1ac1003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac107150 0x7fa1ac1003e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4007f10 tx=0x7fa1a400d3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 shutdown_connections 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa19c0a1500 0x7fa19c0a39c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1ac107150 0x7fa1ac1003e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1ac101ef0 0x7fa1ac1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 --2- 172.21.15.66:0/2859456948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ac104820 0x7fa1ac0ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:51.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.796+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 >> 172.21.15.66:0/2859456948 conn(0x7fa1ac0fb900 msgr2=0x7fa1ac102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.797+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 shutdown_connections 2023-12-11T19:34:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:51.797+0000 7fa1b199d700 1 -- 172.21.15.66:0/2859456948 wait complete. 2023-12-11T19:34:51.799 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-12-11T19:34:51.828 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-11T19:34:52.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:51 smithi167 ceph-mon[102750]: pgmap v398: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:52.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:51 smithi066 ceph-mon[97920]: pgmap v398: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:52.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:51 smithi070 ceph-mon[104044]: pgmap v398: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 -- 172.21.15.66:0/1063639470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8105130 msgr2=0x7f3ca81055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1063639470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8105130 0x7f3ca81055b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b0a0 tx=0x7f3ca000b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 -- 172.21.15.66:0/1063639470 shutdown_connections 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1063639470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ca80ffdf0 0x7f3ca8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1063639470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8105130 0x7f3ca81055b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1063639470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca8102760 0x7f3ca8104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 -- 172.21.15.66:0/1063639470 >> 172.21.15.66:0/1063639470 conn(0x7f3ca80fb9f0 msgr2=0x7f3ca80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 -- 172.21.15.66:0/1063639470 shutdown_connections 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.167+0000 7f3cafdff700 1 -- 172.21.15.66:0/1063639470 wait complete. 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.168+0000 7f3cafdff700 1 Processor -- start 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.168+0000 7f3cafdff700 1 -- start start 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cafdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca80ffdf0 0x7f3ca819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cafdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 0x7f3ca819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cafdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ca8105130 0x7f3ca81a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cafdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81a5680 con 0x7f3ca8102760 2023-12-11T19:34:52.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cafdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81a57c0 con 0x7f3ca8105130 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cafdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81a5950 con 0x7f3ca80ffdf0 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cad39a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 0x7f3ca819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cad39a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 0x7f3ca819e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44608/0 (socket says 172.21.15.66:44608) 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cad39a700 1 -- 172.21.15.66:0/1825854747 learned_addr learned my addr 172.21.15.66:0/1825854747 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca80ffdf0 0x7f3ca819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.169+0000 7f3cae39c700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ca8105130 0x7f3ca81a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cad39a700 1 -- 172.21.15.66:0/1825854747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca80ffdf0 msgr2=0x7f3ca819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cad39a700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca80ffdf0 0x7f3ca819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cad39a700 1 -- 172.21.15.66:0/1825854747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ca8105130 msgr2=0x7f3ca81a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cad39a700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ca8105130 0x7f3ca81a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cad39a700 1 -- 172.21.15.66:0/1825854747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca0009d00 con 0x7f3ca8102760 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca80ffdf0 0x7f3ca819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:52.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cad39a700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 0x7f3ca819e0e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d980 tx=0x7f3c9800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3c9effd700 1 -- 172.21.15.66:0/1825854747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800bb60 con 0x7f3ca8102760 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca81a5c30 con 0x7f3ca8102760 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3c9effd700 1 -- 172.21.15.66:0/1825854747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c98007640 con 0x7f3ca8102760 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3c9effd700 1 -- 172.21.15.66:0/1825854747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98004d60 con 0x7f3ca8102760 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.170+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca81a6190 con 0x7f3ca8102760 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.174+0000 7f3c9effd700 1 -- 172.21.15.66:0/1825854747 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3c98005020 con 0x7f3ca8102760 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 Processor -- start 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 -- start start 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x5644527137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452714640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x5644527150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445266eb50 con 0x5644527133d0 2023-12-11T19:34:52.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445266c470 con 0x564452713d30 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.175+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452667d60 con 0x564452714c20 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3c9effd700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c940a1500 0x7f3c940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3c9effd700 1 -- 172.21.15.66:0/1825854747 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c980ccbd0 con 0x7f3ca8102760 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3cae39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x5644527137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3cae39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x5644527137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44612/0 (socket says 172.21.15.66:44612) 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3cae39c700 1 -- 172.21.15.66:0/795873430 learned_addr learned my addr 172.21.15.66:0/795873430 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452714640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.176+0000 7f3cad39a700 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x5644527150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cadb9b700 1 -- 172.21.15.66:0/795873430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 msgr2=0x5644527150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x5644527150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cadb9b700 1 -- 172.21.15.66:0/795873430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 msgr2=0x5644527137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x5644527137f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cadb9b700 1 -- 172.21.15.66:0/795873430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644524e4630 con 0x564452713d30 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cae39c700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c940a1500 0x7f3c940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452714640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009cc0 tx=0x7f3ca40113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9d4e12426eca2d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3c93fff700 1 -- 172.21.15.66:0/795873430 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4011c30 con 0x564452713d30 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3c93fff700 1 -- 172.21.15.66:0/795873430 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ca4011d90 con 0x564452713d30 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3cae39c700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c940a1500 0x7f3c940a39c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009850 tx=0x7f3ca000b3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.177+0000 7f3c93fff700 1 -- 172.21.15.66:0/795873430 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400f6a0 con 0x564452713d30 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/795873430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 msgr2=0x564452714640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452714640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009cc0 tx=0x7f3ca40113e0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/795873430 shutdown_connections 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x5644527150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x5644527137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/795873430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452714640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/795873430 >> 172.21.15.66:0/795873430 conn(0x5644526427e0 msgr2=0x56445270fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/795873430 shutdown_connections 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.178+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/795873430 wait complete. 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.179+0000 7f3cc36c4b80 1 -- start start 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.179+0000 7f3cc36c4b80 1 Processor -- start 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x56445271d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452724750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452621560 con 0x5644527133d0 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452621200 con 0x564452713d30 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644524e6920 con 0x564452714c20 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cc36c4b80 10 client.0 _ll_register_callbacks cb 0x56445254ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44628/0 (socket says 172.21.15.66:44628) 2023-12-11T19:34:52.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cad39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x56445271d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 -- 172.21.15.66:0/4274363443 learned_addr learned my addr 172.21.15.66:0/4274363443 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cae39c700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452724750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 -- 172.21.15.66:0/4274363443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 msgr2=0x564452724750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452724750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 -- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 msgr2=0x56445271d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.180+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x56445271d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3cadb9b700 1 -- 172.21.15.66:0/4274363443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca40098f0 con 0x5644527133d0 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3cad39a700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x56445271d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271d1d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009600 tx=0x7f3ca40052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ab26d61a604f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3c917fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3c917fa700 1 -- 172.21.15.66:0/4274363443 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40056e0 con 0x5644527133d0 2023-12-11T19:34:52.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564452706b40 con 0x5644527133d0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.181+0000 7f3c917fa700 1 -- 172.21.15.66:0/4274363443 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3ca4004750 con 0x5644527133d0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3c917fa700 1 -- 172.21.15.66:0/4274363443 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca4004910 con 0x5644527133d0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 10 client.15780 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564452490de0 con 0x5644527133d0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3c917fa700 1 -- 172.21.15.66:0/4274363443 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564452706b40 con 0x5644527133d0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3c917fa700 1 -- 172.21.15.66:0/4274363443 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ca4022070 con 0x5644527133d0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 10 client.15780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 20 client.15780 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452728ef0 0x56445272b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 4 client.15780 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.182+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644526e2f40 con 0x564452728ef0 2023-12-11T19:34:52.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.183+0000 7f3cad39a700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452728ef0 0x56445272b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.183+0000 7f3c917fa700 10 client.15780 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.183+0000 7f3cad39a700 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452728ef0 0x56445272b310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6c506cfa7743763 server_cookie=130b92feed914fe6 in_seq=0 out_seq=0 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.184+0000 7f3c917fa700 1 -- 172.21.15.66:0/4274363443 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644526e2f40 con 0x564452728ef0 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.184+0000 7f3c917fa700 10 client.15780 handle_command_reply: tid=0 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.184+0000 7f3cc36c4b80 1 client.15780 shutdown 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 msgr2=0x56445271d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271d1d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009600 tx=0x7f3ca40052e0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 shutdown_connections 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452728ef0 0x56445272b310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452724750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271d1d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3c917fa700 0 client.15780 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:52.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/4274363443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x56445271d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 >> 172.21.15.66:0/4274363443 conn(0x5644526427e0 msgr2=0x56445270ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 shutdown_connections 2023-12-11T19:34:52.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/4274363443 wait complete. 2023-12-11T19:34:52.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.185+0000 7f3cc36c4b80 20 client.15780 trim_cache size 0 max 16384 2023-12-11T19:34:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1352039933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 msgr2=0x7f9ccc109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1352039933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc109670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400b0a0 tx=0x7f9cc400b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1352039933 shutdown_connections 2023-12-11T19:34:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1352039933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc101fa0 0x7f9ccc104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1352039933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc109670 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1352039933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ccc1048d0 0x7f9ccc106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1352039933 >> 172.21.15.66:0/1352039933 conn(0x7f9ccc0fb990 msgr2=0x7f9ccc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.191+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1352039933 shutdown_connections 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.192+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1352039933 wait complete. 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 Processor -- start 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 -- start start 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc101fa0 0x7f9ccc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ccc1048d0 0x7f9ccc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1a11d0 con 0x7f9ccc107200 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1a1310 con 0x7f9ccc101fa0 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9cd0dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1a14a0 con 0x7f9ccc1048d0 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9ccad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9ccad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44634/0 (socket says 172.21.15.66:44634) 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.193+0000 7f9ccad9d700 1 -- 172.21.15.66:0/1451443672 learned_addr learned my addr 172.21.15.66:0/1451443672 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9cca59c700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc101fa0 0x7f9ccc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ccc1048d0 0x7f9ccc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9ccad9d700 1 -- 172.21.15.66:0/1451443672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ccc1048d0 msgr2=0x7f9ccc199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9ccad9d700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ccc1048d0 0x7f9ccc199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9ccad9d700 1 -- 172.21.15.66:0/1451443672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc101fa0 msgr2=0x7f9ccc1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9ccad9d700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc101fa0 0x7f9ccc1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9ccad9d700 1 -- 172.21.15.66:0/1451443672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4009d00 con 0x7f9ccc107200 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9ccad9d700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc1a0c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400b780 tx=0x7f9cc400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.194+0000 7f9cbb7fe700 1 -- 172.21.15.66:0/1451443672 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400e040 con 0x7f9ccc107200 2023-12-11T19:34:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.195+0000 7f9cbb7fe700 1 -- 172.21.15.66:0/1451443672 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cc4003c00 con 0x7f9ccc107200 2023-12-11T19:34:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.195+0000 7f9cbb7fe700 1 -- 172.21.15.66:0/1451443672 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40052a0 con 0x7f9ccc107200 2023-12-11T19:34:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.195+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc1a1720 con 0x7f9ccc107200 2023-12-11T19:34:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.195+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1a1af0 con 0x7f9ccc107200 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.198+0000 7f9cbb7fe700 1 -- 172.21.15.66:0/1451443672 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9cc401d480 con 0x7f9ccc107200 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.199+0000 7f9ce4623b80 1 Processor -- start 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.199+0000 7f9ce4623b80 1 -- start start 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c4072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c407830 0x55ee3c408140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c408ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c362af0 con 0x55ee3c406ed0 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c360410 con 0x55ee3c407830 2023-12-11T19:34:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c35bd00 con 0x55ee3c408720 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9cbb7fe700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cb40a1500 0x7f9cb40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9cbb7fe700 1 -- 172.21.15.66:0/1451443672 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cc40cdcb0 con 0x7f9ccc107200 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c4072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c4072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44650/0 (socket says 172.21.15.66:44650) 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/3585419699 learned_addr learned my addr 172.21.15.66:0/3585419699 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9ccad9d700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cb40a1500 0x7f9cb40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.200+0000 7f9cca59c700 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c407830 0x55ee3c408140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/3585419699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 msgr2=0x55ee3c408ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c408ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/3585419699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c407830 msgr2=0x55ee3c408140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c407830 0x55ee3c408140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/3585419699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3c1d85d0 con 0x55ee3c406ed0 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c4072f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00046e0 tx=0x7f9cc0013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7060b38c77d687a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cb8ff9700 1 -- 172.21.15.66:0/3585419699 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0010430 con 0x55ee3c406ed0 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.201+0000 7f9cb8ff9700 1 -- 172.21.15.66:0/3585419699 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cc0010a20 con 0x55ee3c406ed0 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9cb8ff9700 1 -- 172.21.15.66:0/3585419699 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001b820 con 0x55ee3c406ed0 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 -- 172.21.15.66:0/3585419699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 msgr2=0x55ee3c4072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ccad9d700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cb40a1500 0x7f9cb40a39c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40062a0 tx=0x7f9cc4009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c4072f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00046e0 tx=0x7f9cc0013740 comp rx=0 tx=0).stop 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 -- 172.21.15.66:0/3585419699 shutdown_connections 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c408ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c4072f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/3585419699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c407830 0x55ee3c408140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.202+0000 7f9ce4623b80 1 -- 172.21.15.66:0/3585419699 >> 172.21.15.66:0/3585419699 conn(0x55ee3c336780 msgr2=0x55ee3c403880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.203+0000 7f9ce4623b80 1 -- 172.21.15.66:0/3585419699 shutdown_connections 2023-12-11T19:34:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.203+0000 7f9ce4623b80 1 -- 172.21.15.66:0/3585419699 wait complete. 2023-12-11T19:34:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.203+0000 7f9ce4623b80 1 -- start start 2023-12-11T19:34:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.203+0000 7f9ce4623b80 1 Processor -- start 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c410ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 0x55ee3c411420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c418250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c315500 con 0x55ee3c407830 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c3151a0 con 0x55ee3c406ed0 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c1da8c0 con 0x55ee3c408720 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9ce4623b80 10 client.0 _ll_register_callbacks cb 0x55ee3c240a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 0x55ee3c411420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 0x55ee3c411420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44660/0 (socket says 172.21.15.66:44660) 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9cca59c700 1 -- 172.21.15.66:0/1193261128 learned_addr learned my addr 172.21.15.66:0/1193261128 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c410ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.204+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c418250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cca59c700 1 -- 172.21.15.66:0/1193261128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 msgr2=0x55ee3c418250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cca59c700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c418250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cca59c700 1 -- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 msgr2=0x55ee3c410ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cca59c700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c410ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cca59c700 1 -- 172.21.15.66:0/1193261128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc0009b10 con 0x55ee3c407830 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c410ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cca59c700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 0x55ee3c411420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc017f70 tx=0x7f9cbc00e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abe2496a7b14cbd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/1193261128 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc01c050 con 0x55ee3c407830 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee3c3fa730 con 0x55ee3c407830 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/1193261128 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cbc021600 con 0x55ee3c407830 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.205+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/1193261128 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cbc0217c0 con 0x55ee3c407830 2023-12-11T19:34:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 10 client.15792 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee3c184d80 con 0x55ee3c407830 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/1193261128 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee3c3fa730 con 0x55ee3c407830 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/1193261128 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9cbc02c9e0 con 0x55ee3c407830 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 20 client.15792 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c41cdf0 0x55ee3c41f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 4 client.15792 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.206+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee3c3d6ee0 con 0x55ee3c41cdf0 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.207+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c41cdf0 0x55ee3c41f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.207+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c41cdf0 0x55ee3c41f210 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abf10e360df98f0 server_cookie=eaf58bed11987ca7 in_seq=0 out_seq=0 2023-12-11T19:34:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.207+0000 7f9cb1ffb700 10 client.15792 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:52.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 Processor -- start 2023-12-11T19:34:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/1193261128 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee3c3d6ee0 con 0x55ee3c41cdf0 2023-12-11T19:34:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f9cb1ffb700 10 client.15792 handle_command_reply: tid=0 2023-12-11T19:34:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f9ce4623b80 1 client.15792 shutdown 2023-12-11T19:34:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 msgr2=0x55ee3c411420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 0x55ee3c411420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc017f70 tx=0x7f9cbc00e740 comp rx=0 tx=0).stop 2023-12-11T19:34:52.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 -- start start 2023-12-11T19:34:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 0x564452724e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452713d30 0x564452724710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564452714c20 0x564452715000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445271ae70 con 0x564452714c20 2023-12-11T19:34:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.208+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452718760 con 0x5644527133d0 2023-12-11T19:34:52.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644527068d0 con 0x564452713d30 2023-12-11T19:34:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 shutdown_connections 2023-12-11T19:34:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c41cdf0 0x55ee3c41f210 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c410ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c408720 0x55ee3c418250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1193261128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c407830 0x55ee3c411420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9cb1ffb700 0 client.15792 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 >> 172.21.15.66:0/1193261128 conn(0x55ee3c336780 msgr2=0x55ee3c401ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 0x564452724e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 shutdown_connections 2023-12-11T19:34:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 0x564452724e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:56098/0 (socket says 172.21.15.66:56098) 2023-12-11T19:34:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 -- 172.21.15.66:0/461769120 learned_addr learned my addr 172.21.15.66:0/461769120 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cae39c700 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564452714c20 0x564452715000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cad39a700 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452713d30 0x564452724710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1193261128 wait complete. 2023-12-11T19:34:52.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 -- 172.21.15.66:0/461769120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452713d30 msgr2=0x564452724710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452713d30 0x564452724710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 -- 172.21.15.66:0/461769120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564452714c20 msgr2=0x564452715000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.209+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564452714c20 0x564452715000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f9ce4623b80 20 client.15792 trim_cache size 0 max 16384 2023-12-11T19:34:52.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f3cadb9b700 1 -- 172.21.15.66:0/461769120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564452479c60 con 0x5644527133d0 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f3cad39a700 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452713d30 0x564452724710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 0x564452724e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009be0 tx=0x7f3ca4013ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac3c742fb8f0a217 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f3c9cbff700 1 -- 172.21.15.66:0/461769120 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4019d50 con 0x5644527133d0 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f3c9cbff700 1 -- 172.21.15.66:0/461769120 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ca4019eb0 con 0x5644527133d0 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.210+0000 7f3c9cbff700 1 -- 172.21.15.66:0/461769120 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4019600 con 0x5644527133d0 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/461769120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 msgr2=0x564452724e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 0x564452724e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009be0 tx=0x7f3ca4013ef0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/461769120 shutdown_connections 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5644527133d0 0x564452724e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564452714c20 0x564452715000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/461769120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452713d30 0x564452724710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/461769120 >> 172.21.15.66:0/461769120 conn(0x5644526427e0 msgr2=0x56445271c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/461769120 shutdown_connections 2023-12-11T19:34:52.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.211+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/461769120 wait complete. 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 -- start start 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 Processor -- start 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452720240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452723260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644527237a0 con 0x5644527133d0 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452723910 con 0x564452713d30 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452723ad0 con 0x564452714c20 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.212+0000 7f3cc36c4b80 10 client.0 _ll_register_callbacks cb 0x56445254ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452720240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cadb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452720240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:56102/0 (socket says 172.21.15.66:56102) 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 -- 172.21.15.66:0/303361578 learned_addr learned my addr 172.21.15.66:0/303361578 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 -- 172.21.15.66:0/303361578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 msgr2=0x564452723260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452723260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 -- 172.21.15.66:0/303361578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 msgr2=0x56445271fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 -- 172.21.15.66:0/303361578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564452490de0 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3cad39a700 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452720240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d950 tx=0x7f3c9800dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac8764a327d6fe2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.213+0000 7f3c917fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3c917fa700 1 -- 172.21.15.66:0/303361578 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98099eb0 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3c917fa700 1 -- 172.21.15.66:0/303361578 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c9809e000 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ca4011d90 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3c917fa700 1 -- 172.21.15.66:0/303361578 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c9809f0c0 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3cc36c4b80 10 client.25547 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5644524b7250 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3c917fa700 1 -- 172.21.15.66:0/303361578 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ca4011d90 con 0x564452713d30 2023-12-11T19:34:52.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3c917fa700 1 -- 172.21.15.66:0/303361578 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c9809a110 con 0x564452713d30 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3cc36c4b80 10 client.25547 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3cc36c4b80 10 client.25547 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.214+0000 7f3cc36c4b80 20 client.25547 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.215+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452790aa0 0x564452792e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.215+0000 7f3cc36c4b80 4 client.25547 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.215+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56445262dfd0 con 0x564452790aa0 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.215+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452790aa0 0x564452792e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.215+0000 7f3cadb9b700 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452790aa0 0x564452792e90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b80f6a4117d6c3c7 server_cookie=de9bc7100fe8711a in_seq=0 out_seq=0 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.215+0000 7f3c917fa700 10 client.25547 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3c917fa700 1 -- 172.21.15.66:0/303361578 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56445262dfd0 con 0x564452790aa0 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3c917fa700 10 client.25547 handle_command_reply: tid=0 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 client.25547 shutdown 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 msgr2=0x564452720240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452720240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d950 tx=0x7f3c9800dea0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 shutdown_connections 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x564452790aa0 0x564452792e90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564452714c20 0x564452723260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644527133d0 0x56445271fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 --2- 172.21.15.66:0/303361578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564452713d30 0x564452720240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3c917fa700 0 client.25547 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 >> 172.21.15.66:0/303361578 conn(0x5644526427e0 msgr2=0x56445271afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 shutdown_connections 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 1 -- 172.21.15.66:0/303361578 wait complete. 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.216+0000 7f3cc36c4b80 20 client.25547 trim_cache size 0 max 16384 2023-12-11T19:34:52.227 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c940a1500 msgr2=0x7f3c940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c940a1500 0x7f3c940a39c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009850 tx=0x7f3ca000b3b0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 msgr2=0x7f3ca819e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 0x7f3ca819e0e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800d980 tx=0x7f3c9800dd40 comp rx=0 tx=0).stop 2023-12-11T19:34:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 shutdown_connections 2023-12-11T19:34:52.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3c940a1500 0x7f3c940a39c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ca8105130 0x7f3ca81a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca80ffdf0 0x7f3ca819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 --2- 172.21.15.66:0/1825854747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca8102760 0x7f3ca819e0e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 >> 172.21.15.66:0/1825854747 conn(0x7f3ca80fb9f0 msgr2=0x7f3ca8100790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 shutdown_connections 2023-12-11T19:34:52.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.217+0000 7f3cafdff700 1 -- 172.21.15.66:0/1825854747 wait complete. 2023-12-11T19:34:52.237 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.237+0000 7f9ce4623b80 1 Processor -- start 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 -- start start 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c4101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c418210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 0x55ee3c408b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c40e8c0 con 0x55ee3c408720 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c41cb90 con 0x55ee3c406ed0 2023-12-11T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c40c1b0 con 0x55ee3c407830 2023-12-11T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 0x55ee3c408b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 0x55ee3c408b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44690/0 (socket says 172.21.15.66:44690) 2023-12-11T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/1136438826 learned_addr learned my addr 172.21.15.66:0/1136438826 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.238+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c4101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cca59c700 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c418210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/1136438826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 msgr2=0x55ee3c418210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c418210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/1136438826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 msgr2=0x55ee3c4101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c4101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cc9d9b700 1 -- 172.21.15.66:0/1136438826 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee3c16dc00 con 0x55ee3c408720 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cca59c700 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c418210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.239+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 0x55ee3c408b00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00046b0 tx=0x7f9cc0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd411b2705dd012 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.240+0000 7f9cb97fa700 1 -- 172.21.15.66:0/1136438826 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001f040 con 0x55ee3c408720 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.240+0000 7f9cb97fa700 1 -- 172.21.15.66:0/1136438826 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cc0032070 con 0x55ee3c408720 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.240+0000 7f9cb97fa700 1 -- 172.21.15.66:0/1136438826 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00053c0 con 0x55ee3c408720 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.240+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1136438826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 msgr2=0x55ee3c408b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.240+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 0x55ee3c408b00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00046b0 tx=0x7f9cc0009920 comp rx=0 tx=0).stop 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1136438826 shutdown_connections 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c406ed0 0x55ee3c4101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c408720 0x55ee3c408b00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/1136438826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c418210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1136438826 >> 172.21.15.66:0/1136438826 conn(0x55ee3c336780 msgr2=0x55ee3c40ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1136438826 shutdown_connections 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 -- 172.21.15.66:0/1136438826 wait complete. 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.241+0000 7f9ce4623b80 1 -- start start 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 Processor -- start 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c416050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c416590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c408720 0x55ee3c4174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c4179f0 con 0x55ee3c406ed0 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c417b60 con 0x55ee3c408720 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3c417d20 con 0x55ee3c407830 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.242+0000 7f9ce4623b80 10 client.0 _ll_register_callbacks cb 0x55ee3c240a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c416590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c416590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:40704/0 (socket says 172.21.15.66:40704) 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c416050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 -- 172.21.15.66:0/2192674717 learned_addr learned my addr 172.21.15.66:0/2192674717 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 -- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c408720 msgr2=0x55ee3c4174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c408720 0x55ee3c4174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 -- 172.21.15.66:0/2192674717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 msgr2=0x55ee3c416050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c416050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.243+0000 7f9cca59c700 1 -- 172.21.15.66:0/2192674717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee3c184d80 con 0x55ee3c407830 2023-12-11T19:34:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9cca59c700 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c416590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02ce00 tx=0x7f9cbc005820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db634c8d3ae7d3b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9cb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/2192674717 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc01c070 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/2192674717 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cbc000e40 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc0010a20 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/2192674717 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cbc02c430 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9ce4623b80 10 client.25546 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.244+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9cc0022040 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/2192674717 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cc0010a20 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/2192674717 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9cbc003a90 con 0x55ee3c407830 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9ce4623b80 10 client.25546 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9ce4623b80 10 client.25546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9ce4623b80 20 client.25546 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c4742d0 0x55ee3c4766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9ce4623b80 4 client.25546 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:34:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.245+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee3c321f70 con 0x55ee3c4742d0 2023-12-11T19:34:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.246+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c4742d0 0x55ee3c4766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.247+0000 7f9cb1ffb700 10 client.25546 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.247+0000 7f9cc9d9b700 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c4742d0 0x55ee3c4766f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a4b2fc69350578 server_cookie=500d5fd97beb5a15 in_seq=0 out_seq=0 2023-12-11T19:34:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9cb1ffb700 1 -- 172.21.15.66:0/2192674717 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee3c321f70 con 0x55ee3c4742d0 2023-12-11T19:34:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9cb1ffb700 10 client.25546 handle_command_reply: tid=0 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 client.25546 shutdown 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 msgr2=0x55ee3c416590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c416590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02ce00 tx=0x7f9cbc005820 comp rx=0 tx=0).stop 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 shutdown_connections 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ee3c4742d0 0x55ee3c4766f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ee3c408720 0x55ee3c4174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ee3c406ed0 0x55ee3c416050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9cb1ffb700 0 client.25546 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 --2- 172.21.15.66:0/2192674717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee3c407830 0x55ee3c416590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.251+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 >> 172.21.15.66:0/2192674717 conn(0x55ee3c336780 msgr2=0x55ee3c40f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 shutdown_connections 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9ce4623b80 1 -- 172.21.15.66:0/2192674717 wait complete. 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9ce4623b80 20 client.25546 trim_cache size 0 max 16384 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:34:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cb40a1500 msgr2=0x7f9cb40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cb40a1500 0x7f9cb40a39c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40062a0 tx=0x7f9cc4009040 comp rx=0 tx=0).stop 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 msgr2=0x7f9ccc1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.252+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc1a0c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400b780 tx=0x7f9cc400bab0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 shutdown_connections 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cb40a1500 0x7f9cb40a39c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc101fa0 0x7f9ccc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ccc107200 0x7f9ccc1a0c00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 --2- 172.21.15.66:0/1451443672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ccc1048d0 0x7f9ccc199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 >> 172.21.15.66:0/1451443672 conn(0x7f9ccc0fb990 msgr2=0x7f9ccc1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 shutdown_connections 2023-12-11T19:34:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:34:52.253+0000 7f9cd0dff700 1 -- 172.21.15.66:0/1451443672 wait complete. 2023-12-11T19:34:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 -- 172.21.15.66:0/1766277241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978075610 msgr2=0x7fc978073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/1766277241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978075610 0x7fc978073a60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc968009990 tx=0x7fc968009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 -- 172.21.15.66:0/1766277241 shutdown_connections 2023-12-11T19:34:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/1766277241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc978109460 0x7fc97810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/1766277241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978075610 0x7fc978073a60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/1766277241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 0x7fc9780744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 -- 172.21.15.66:0/1766277241 >> 172.21.15.66:0/1766277241 conn(0x7fc9780fcb10 msgr2=0x7fc9780fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 -- 172.21.15.66:0/1766277241 shutdown_connections 2023-12-11T19:34:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.459+0000 7fc97e3ff700 1 -- 172.21.15.66:0/1766277241 wait complete. 2023-12-11T19:34:52.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.460+0000 7fc97e3ff700 1 Processor -- start 2023-12-11T19:34:52.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.460+0000 7fc97e3ff700 1 -- start start 2023-12-11T19:34:52.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.460+0000 7fc97e3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 0x7fc9781996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.460+0000 7fc97e3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc978075610 0x7fc978199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97e3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 0x7fc9781a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97e3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9781a1220 con 0x7fc978109460 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97e3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9781a1360 con 0x7fc978075610 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97e3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9781a14f0 con 0x7fc978074030 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 0x7fc9781a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 0x7fc9781a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44716/0 (socket says 172.21.15.66:44716) 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 -- 172.21.15.66:0/414646545 learned_addr learned my addr 172.21.15.66:0/414646545 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc977fff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 0x7fc9781996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc9777fe700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc978075610 0x7fc978199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 -- 172.21.15.66:0/414646545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 msgr2=0x7fc9781996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 0x7fc9781996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 -- 172.21.15.66:0/414646545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc978075610 msgr2=0x7fc978199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc978075610 0x7fc978199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc97c99c700 1 -- 172.21.15.66:0/414646545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680095f0 con 0x7fc978109460 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.461+0000 7fc977fff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 0x7fc9781996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:52.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.462+0000 7fc97c99c700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 0x7fc9781a0c50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc97000b970 tx=0x7fc97000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.462+0000 7fc9757fa700 1 -- 172.21.15.66:0/414646545 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97000c9b0 con 0x7fc978109460 2023-12-11T19:34:52.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.462+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9781a17d0 con 0x7fc978109460 2023-12-11T19:34:52.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.462+0000 7fc9757fa700 1 -- 172.21.15.66:0/414646545 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc970014440 con 0x7fc978109460 2023-12-11T19:34:52.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.462+0000 7fc9757fa700 1 -- 172.21.15.66:0/414646545 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700075d0 con 0x7fc978109460 2023-12-11T19:34:52.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.462+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9781a1c90 con 0x7fc978109460 2023-12-11T19:34:52.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.467+0000 7fc9757fa700 1 -- 172.21.15.66:0/414646545 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc9700145b0 con 0x7fc978109460 2023-12-11T19:34:52.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.468+0000 7fc991cccb80 1 Processor -- start 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.468+0000 7fc991cccb80 1 -- start start 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.468+0000 7fc9757fa700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9600a1480 0x7fc9600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc9757fa700 1 -- 172.21.15.66:0/414646545 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc970006e40 con 0x7fc978109460 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc9777fe700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9600a1480 0x7fc9600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da43d0 0x55e6e6da47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6da5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc991cccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6cffb50 con 0x55e6e6da5c20 2023-12-11T19:34:52.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc991cccb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6cfd470 con 0x55e6e6da4d30 2023-12-11T19:34:52.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc991cccb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6cf8d60 con 0x55e6e6da43d0 2023-12-11T19:34:52.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc97c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc97c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44724/0 (socket says 172.21.15.66:44724) 2023-12-11T19:34:52.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc97c99c700 1 -- 172.21.15.66:0/3346663072 learned_addr learned my addr 172.21.15.66:0/3346663072 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc977fff700 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da43d0 0x55e6e6da47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.469+0000 7fc9777fe700 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6da5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc97c99c700 1 -- 172.21.15.66:0/3346663072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da43d0 msgr2=0x55e6e6da47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc97c99c700 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da43d0 0x55e6e6da47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc97c99c700 1 -- 172.21.15.66:0/3346663072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 msgr2=0x55e6e6da5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc97c99c700 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6da5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc9777fe700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9600a1480 0x7fc9600a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc96c005950 tx=0x7fc96c00f640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc97c99c700 1 -- 172.21.15.66:0/3346663072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6e6b75630 con 0x55e6e6da5c20 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc97c99c700 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da60a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc970096350 tx=0x7fc970099240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a941988d7e2d64b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc95e7fc700 1 -- 172.21.15.66:0/3346663072 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970099fe0 con 0x55e6e6da5c20 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc95e7fc700 1 -- 172.21.15.66:0/3346663072 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9700998e0 con 0x55e6e6da5c20 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.470+0000 7fc95e7fc700 1 -- 172.21.15.66:0/3346663072 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970098dd0 con 0x55e6e6da5c20 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 -- 172.21.15.66:0/3346663072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 msgr2=0x55e6e6da60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da60a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc970096350 tx=0x7fc970099240 comp rx=0 tx=0).stop 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 -- 172.21.15.66:0/3346663072 shutdown_connections 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da60a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da43d0 0x55e6e6da47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3346663072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6da5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 -- 172.21.15.66:0/3346663072 >> 172.21.15.66:0/3346663072 conn(0x55e6e6cd37e0 msgr2=0x55e6e6da0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 -- 172.21.15.66:0/3346663072 shutdown_connections 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.471+0000 7fc991cccb80 1 -- 172.21.15.66:0/3346663072 wait complete. 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 -- start start 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 Processor -- start 2023-12-11T19:34:52.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6dae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6dae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da5c20 0x55e6e6db5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6cb2560 con 0x55e6e6da43d0 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6cb2200 con 0x55e6e6da4d30 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.472+0000 7fc991cccb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6b77920 con 0x55e6e6da5c20 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc991cccb80 10 client.0 _ll_register_callbacks cb 0x55e6e6bdda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6dae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6dae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44732/0 (socket says 172.21.15.66:44732) 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 -- 172.21.15.66:0/1198991581 learned_addr learned my addr 172.21.15.66:0/1198991581 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc9777fe700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6dae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc97c99c700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da5c20 0x55e6e6db5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 -- 172.21.15.66:0/1198991581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da5c20 msgr2=0x55e6e6db5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da5c20 0x55e6e6db5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 -- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 msgr2=0x55e6e6dae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6dae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc977fff700 1 -- 172.21.15.66:0/1198991581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9781a1c90 con 0x55e6e6da43d0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.473+0000 7fc97c99c700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da5c20 0x55e6e6db5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc977fff700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6dae210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc968009990 tx=0x7fc968005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b764abe18710dcfd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc9577fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc9577fe700 1 -- 172.21.15.66:0/1198991581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968009ee0 con 0x55e6e6da43d0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6e6d97b40 con 0x55e6e6da43d0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc9577fe700 1 -- 172.21.15.66:0/1198991581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc968004500 con 0x55e6e6da43d0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc9577fe700 1 -- 172.21.15.66:0/1198991581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc968005720 con 0x55e6e6da43d0 2023-12-11T19:34:52.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc991cccb80 10 client.15816 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6e6b21de0 con 0x55e6e6da43d0 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.474+0000 7fc9577fe700 1 -- 172.21.15.66:0/1198991581 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc968033020 con 0x55e6e6da43d0 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc9577fe700 1 -- 172.21.15.66:0/1198991581 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc968003a80 con 0x55e6e6da43d0 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc991cccb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc991cccb80 10 client.15816 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc991cccb80 20 client.15816 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc991cccb80 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6dba060 0x55e6e6dbc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc991cccb80 4 client.15816 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6e6d73f40 con 0x55e6e6dba060 2023-12-11T19:34:52.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.475+0000 7fc9777fe700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6dba060 0x55e6e6dbc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.476+0000 7fc9577fe700 10 client.15816 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:52.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.476+0000 7fc9777fe700 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6dba060 0x55e6e6dbc480 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95ab2a3eda1ad847 server_cookie=e58079fb15922f8 in_seq=0 out_seq=0 2023-12-11T19:34:52.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.476+0000 7fc9577fe700 1 -- 172.21.15.66:0/1198991581 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6e6d73f40 con 0x55e6e6dba060 2023-12-11T19:34:52.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.476+0000 7fc9577fe700 10 client.15816 handle_command_reply: tid=0 2023-12-11T19:34:52.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 client.15816 shutdown 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 msgr2=0x55e6e6dae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6dae210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc968009990 tx=0x7fc968005270 comp rx=0 tx=0).stop 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 shutdown_connections 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da5c20 0x55e6e6db5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6dae210 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6dba060 0x55e6e6dbc480 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 --2- 172.21.15.66:0/1198991581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da4d30 0x55e6e6dae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc9577fe700 0 client.15816 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 >> 172.21.15.66:0/1198991581 conn(0x55e6e6cd37e0 msgr2=0x55e6e6d9fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 shutdown_connections 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 1 -- 172.21.15.66:0/1198991581 wait complete. 2023-12-11T19:34:52.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.477+0000 7fc991cccb80 20 client.15816 trim_cache size 0 max 16384 2023-12-11T19:34:52.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 Processor -- start 2023-12-11T19:34:52.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 -- start start 2023-12-11T19:34:52.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 0x55e6e6da6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6d9b790 con 0x55e6e6da5c20 2023-12-11T19:34:52.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6db6910 con 0x55e6e6da43d0 2023-12-11T19:34:52.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.492+0000 7fc991cccb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6dc16b0 con 0x55e6e6da4d30 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc977fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 0x55e6e6da6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc9777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc977fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 0x55e6e6da6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:56170/0 (socket says 172.21.15.66:56170) 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc97c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc977fff700 1 -- 172.21.15.66:0/771862835 learned_addr learned my addr 172.21.15.66:0/771862835 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc9777fe700 1 -- 172.21.15.66:0/771862835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 msgr2=0x55e6e6da6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc9777fe700 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 0x55e6e6da6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc9777fe700 1 -- 172.21.15.66:0/771862835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 msgr2=0x55e6e6da0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc9777fe700 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc9777fe700 1 -- 172.21.15.66:0/771862835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6e6b0ac60 con 0x55e6e6da4d30 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc977fff700 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 0x55e6e6da6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.493+0000 7fc97c99c700 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.494+0000 7fc9777fe700 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db5750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55e6e6dbcd80 tx=0x7fc96c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e6794c4ab13ebea server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.494+0000 7fc95effd700 1 -- 172.21.15.66:0/771862835 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c023040 con 0x55e6e6da4d30 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.494+0000 7fc95effd700 1 -- 172.21.15.66:0/771862835 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc96c0283d0 con 0x55e6e6da4d30 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.494+0000 7fc95effd700 1 -- 172.21.15.66:0/771862835 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c011800 con 0x55e6e6da4d30 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 -- 172.21.15.66:0/771862835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 msgr2=0x55e6e6db5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db5750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55e6e6dbcd80 tx=0x7fc96c014000 comp rx=0 tx=0).stop 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 -- 172.21.15.66:0/771862835 shutdown_connections 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da43d0 0x55e6e6da6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da5c20 0x55e6e6da0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 --2- 172.21.15.66:0/771862835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db5750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 -- 172.21.15.66:0/771862835 >> 172.21.15.66:0/771862835 conn(0x55e6e6cd37e0 msgr2=0x55e6e6dad3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 -- 172.21.15.66:0/771862835 shutdown_connections 2023-12-11T19:34:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 -- 172.21.15.66:0/771862835 wait complete. 2023-12-11T19:34:52.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.495+0000 7fc991cccb80 1 -- start start 2023-12-11T19:34:52.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 Processor -- start 2023-12-11T19:34:52.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6db5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da5c20 0x55e6e6db3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6db5a60 con 0x55e6e6da43d0 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6db4320 con 0x55e6e6da5c20 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6e6db44e0 con 0x55e6e6da4d30 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.496+0000 7fc991cccb80 10 client.0 _ll_register_callbacks cb 0x55e6e6bdda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6db5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6db5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44746/0 (socket says 172.21.15.66:44746) 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 -- 172.21.15.66:0/3333049798 learned_addr learned my addr 172.21.15.66:0/3333049798 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc97c99c700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da5c20 0x55e6e6db3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc9777fe700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 -- 172.21.15.66:0/3333049798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 msgr2=0x55e6e6db38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 -- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da5c20 msgr2=0x55e6e6db3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da5c20 0x55e6e6db3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 -- 172.21.15.66:0/3333049798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6e6b21de0 con 0x55e6e6da43d0 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc977fff700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6db5240 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c010 tx=0x7fc968009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb54afdf7c7d0273 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.497+0000 7fc9577fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc9577fe700 1 -- 172.21.15.66:0/3333049798 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968003c30 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc9577fe700 1 -- 172.21.15.66:0/3333049798 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc968034080 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9700998e0 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc9577fe700 1 -- 172.21.15.66:0/3333049798 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc968004500 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc991cccb80 10 client.15822 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e6e6b75630 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.498+0000 7fc9577fe700 1 -- 172.21.15.66:0/3333049798 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9700998e0 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc9577fe700 1 -- 172.21.15.66:0/3333049798 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc968036040 con 0x55e6e6da43d0 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc991cccb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc991cccb80 10 client.15822 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc991cccb80 20 client.15822 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6e116c0 0x55e6e6e13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc991cccb80 4 client.15822 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6e6cbefd0 con 0x55e6e6e116c0 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.499+0000 7fc9777fe700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6e116c0 0x55e6e6e13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc9777fe700 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6e116c0 0x55e6e6e13ae0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa8782a8c30c12f8 server_cookie=4bf4918f1b2f94d8 in_seq=0 out_seq=0 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc9577fe700 10 client.15822 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc9577fe700 1 -- 172.21.15.66:0/3333049798 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6e6cbefd0 con 0x55e6e6e116c0 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc9577fe700 10 client.15822 handle_command_reply: tid=0 2023-12-11T19:34:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 client.15822 shutdown 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 msgr2=0x55e6e6db5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6db5240 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c010 tx=0x7fc968009f50 comp rx=0 tx=0).stop 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 shutdown_connections 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55e6e6e116c0 0x55e6e6e13ae0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e6e6da5c20 0x55e6e6db3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6e6da43d0 0x55e6e6db5240 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 --2- 172.21.15.66:0/3333049798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6e6da4d30 0x55e6e6db38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc9577fe700 0 client.15822 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:34:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 >> 172.21.15.66:0/3333049798 conn(0x55e6e6cd37e0 msgr2=0x55e6e6da0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.500+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 shutdown_connections 2023-12-11T19:34:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.501+0000 7fc991cccb80 1 -- 172.21.15.66:0/3333049798 wait complete. 2023-12-11T19:34:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.501+0000 7fc991cccb80 20 client.15822 trim_cache size 0 max 16384 2023-12-11T19:34:52.508 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:52.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.501+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9600a1480 msgr2=0x7fc9600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.501+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9600a1480 0x7fc9600a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc96c005950 tx=0x7fc96c00f640 comp rx=0 tx=0).stop 2023-12-11T19:34:52.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.501+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 msgr2=0x7fc9781a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.501+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 0x7fc9781a0c50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc97000b970 tx=0x7fc97000bd30 comp rx=0 tx=0).stop 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 shutdown_connections 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc9600a1480 0x7fc9600a3940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc978109460 0x7fc9781a0c50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc978074030 0x7fc9781996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 --2- 172.21.15.66:0/414646545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc978075610 0x7fc978199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 >> 172.21.15.66:0/414646545 conn(0x7fc9780fcb10 msgr2=0x7fc978105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 shutdown_connections 2023-12-11T19:34:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.502+0000 7fc97e3ff700 1 -- 172.21.15.66:0/414646545 wait complete. 2023-12-11T19:34:52.529 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 -- 172.21.15.66:0/4067753004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38201037b0 msgr2=0x7f3820103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 --2- 172.21.15.66:0/4067753004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38201037b0 0x7f3820103bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f381c009990 tx=0x7f381c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 -- 172.21.15.66:0/4067753004 shutdown_connections 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 --2- 172.21.15.66:0/4067753004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3820105340 0x7f382010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 --2- 172.21.15.66:0/4067753004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38201037b0 0x7f3820103bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 --2- 172.21.15.66:0/4067753004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201049a0 0x7f3820104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.790+0000 7f38279ff700 1 -- 172.21.15.66:0/4067753004 >> 172.21.15.66:0/4067753004 conn(0x7f38200fed20 msgr2=0x7f3820101160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.791+0000 7f38279ff700 1 -- 172.21.15.66:0/4067753004 shutdown_connections 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.791+0000 7f38279ff700 1 -- 172.21.15.66:0/4067753004 wait complete. 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 Processor -- start 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 -- start start 2023-12-11T19:34:52.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201037b0 0x7f3820199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38201049a0 0x7f3820199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3820105340 0x7f38201a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38201a1200 con 0x7f38201049a0 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38201a1340 con 0x7f38201037b0 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f38279ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38201a14d0 con 0x7f3820105340 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f3824f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38201049a0 0x7f3820199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.792+0000 7f382579b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201037b0 0x7f3820199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f3824f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38201049a0 0x7f3820199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44770/0 (socket says 172.21.15.66:44770) 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f3824f9a700 1 -- 172.21.15.66:0/327258710 learned_addr learned my addr 172.21.15.66:0/327258710 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f3825f9c700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3820105340 0x7f38201a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f382579b700 1 -- 172.21.15.66:0/327258710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3820105340 msgr2=0x7f38201a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f382579b700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3820105340 0x7f38201a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f382579b700 1 -- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38201049a0 msgr2=0x7f3820199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f382579b700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38201049a0 0x7f3820199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f382579b700 1 -- 172.21.15.66:0/327258710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0095f0 con 0x7f38201037b0 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.793+0000 7f382579b700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201037b0 0x7f3820199720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f381c0098e0 tx=0x7f381c00b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.794+0000 7f38167fc700 1 -- 172.21.15.66:0/327258710 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c01d070 con 0x7f38201037b0 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.794+0000 7f38167fc700 1 -- 172.21.15.66:0/327258710 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f381c003d20 con 0x7f38201037b0 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.794+0000 7f38167fc700 1 -- 172.21.15.66:0/327258710 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c022cb0 con 0x7f38201037b0 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.794+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38201a1750 con 0x7f38201037b0 2023-12-11T19:34:52.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.794+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38201a1b20 con 0x7f38201037b0 2023-12-11T19:34:52.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.797+0000 7f38167fc700 1 -- 172.21.15.66:0/327258710 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f381c003890 con 0x7f38201037b0 2023-12-11T19:34:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.799+0000 7f38167fc700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f380c0a1500 0x7f380c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.799+0000 7f38167fc700 1 -- 172.21.15.66:0/327258710 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f381c0ce260 con 0x7f38201037b0 2023-12-11T19:34:52.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.799+0000 7f3824f9a700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f380c0a1500 0x7f380c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.800+0000 7f3824f9a700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f380c0a1500 0x7f380c0a39c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3810009d20 tx=0x7f3810009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.801+0000 7f383b2c6b80 1 Processor -- start 2023-12-11T19:34:52.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.801+0000 7f383b2c6b80 1 -- start start 2023-12-11T19:34:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bb6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bb70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0b10b50 con 0x557dc0bb5d30 2023-12-11T19:34:52.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0b0e470 con 0x557dc0bb6c20 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0b09d60 con 0x557dc0bb53d0 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bb6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bb6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44780/0 (socket says 172.21.15.66:44780) 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 -- 172.21.15.66:0/2396073367 learned_addr learned my addr 172.21.15.66:0/2396073367 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f382579b700 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3825f9c700 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bb70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 -- 172.21.15.66:0/2396073367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 msgr2=0x557dc0bb57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 -- 172.21.15.66:0/2396073367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 msgr2=0x557dc0bb70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bb70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3824f9a700 1 -- 172.21.15.66:0/2396073367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc0986630 con 0x557dc0bb5d30 2023-12-11T19:34:52.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.802+0000 7f3825f9c700 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bb70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.803+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bb6640 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f381000e540 tx=0x7f3810014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31b9709c4474056 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.803+0000 7f380b7fe700 1 -- 172.21.15.66:0/2396073367 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810026430 con 0x557dc0bb5d30 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.803+0000 7f380b7fe700 1 -- 172.21.15.66:0/2396073367 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3810021070 con 0x557dc0bb5d30 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.803+0000 7f380b7fe700 1 -- 172.21.15.66:0/2396073367 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381001d6e0 con 0x557dc0bb5d30 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2396073367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 msgr2=0x557dc0bb6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bb6640 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f381000e540 tx=0x7f3810014000 comp rx=0 tx=0).stop 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2396073367 shutdown_connections 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bb70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2396073367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bb6640 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2396073367 >> 172.21.15.66:0/2396073367 conn(0x557dc0ae47e0 msgr2=0x557dc0bb1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2396073367 shutdown_connections 2023-12-11T19:34:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.804+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2396073367 wait complete. 2023-12-11T19:34:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.805+0000 7f383b2c6b80 1 -- start start 2023-12-11T19:34:52.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.805+0000 7f383b2c6b80 1 Processor -- start 2023-12-11T19:34:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.805+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 0x557dc0bbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.805+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bbf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb6c20 0x557dc0bc6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0ac3560 con 0x557dc0bb53d0 2023-12-11T19:34:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0ac3200 con 0x557dc0bb5d30 2023-12-11T19:34:52.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0988920 con 0x557dc0bb6c20 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f383b2c6b80 10 client.0 _ll_register_callbacks cb 0x557dc09eea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 0x557dc0bbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 0x557dc0bbf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44782/0 (socket says 172.21.15.66:44782) 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 -- 172.21.15.66:0/2594497692 learned_addr learned my addr 172.21.15.66:0/2594497692 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f3825f9c700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb6c20 0x557dc0bc6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bbf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 -- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb6c20 msgr2=0x557dc0bc6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb6c20 0x557dc0bc6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 -- 172.21.15.66:0/2594497692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 msgr2=0x557dc0bbf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bbf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f382579b700 1 -- 172.21.15.66:0/2594497692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3810017070 con 0x557dc0bb53d0 2023-12-11T19:34:52.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.806+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f382579b700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 0x557dc0bbf200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f381c009990 tx=0x7f381c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf97a8749d4974b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f3808ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f3808ff9700 1 -- 172.21.15.66:0/2594497692 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c022470 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f3808ff9700 1 -- 172.21.15.66:0/2594497692 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f381c003b40 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557dc0ba8b40 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f3808ff9700 1 -- 172.21.15.66:0/2594497692 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f381c09bcd0 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f383b2c6b80 10 client.15834 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557dc0932de0 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.807+0000 7f3808ff9700 1 -- 172.21.15.66:0/2594497692 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f381c0b7080 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f3808ff9700 1 -- 172.21.15.66:0/2594497692 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f381c0a3440 con 0x557dc0bb53d0 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f383b2c6b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f383b2c6b80 10 client.15834 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f383b2c6b80 20 client.15834 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0bcab40 0x557dc0bccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f383b2c6b80 4 client.15834 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:52.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.808+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dc0b84f40 con 0x557dc0bcab40 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.809+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0bcab40 0x557dc0bccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.809+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0bcab40 0x557dc0bccf30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9334b51f05ed9a80 server_cookie=89b78d4dbc5fd9b5 in_seq=0 out_seq=0 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.809+0000 7f3808ff9700 10 client.15834 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f3808ff9700 1 -- 172.21.15.66:0/2594497692 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557dc0b84f40 con 0x557dc0bcab40 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f3808ff9700 10 client.15834 handle_command_reply: tid=0 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 client.15834 shutdown 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 msgr2=0x557dc0bbf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 0x557dc0bbf200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f381c009990 tx=0x7f381c00bfb0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 shutdown_connections 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0bcab40 0x557dc0bccf30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb6c20 0x557dc0bc6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb53d0 0x557dc0bbf200 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f3808ff9700 0 client.15834 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2594497692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.810+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 >> 172.21.15.66:0/2594497692 conn(0x557dc0ae47e0 msgr2=0x557dc0bb0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.811+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 shutdown_connections 2023-12-11T19:34:52.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.811+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2594497692 wait complete. 2023-12-11T19:34:52.814 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:52 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2859456948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:34:52.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.811+0000 7f383b2c6b80 20 client.15834 trim_cache size 0 max 16384 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 Processor -- start 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 -- start start 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bc6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bbdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0bb5090 con 0x557dc0bb5d30 2023-12-11T19:34:52.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0bbeda0 con 0x557dc0bb6c20 2023-12-11T19:34:52.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.825+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0bc7950 con 0x557dc0bb53d0 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f3824f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bc6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f3824f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bc6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44798/0 (socket says 172.21.15.66:44798) 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f3824f9a700 1 -- 172.21.15.66:0/2499349381 learned_addr learned my addr 172.21.15.66:0/2499349381 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f3825f9c700 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bbdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 -- 172.21.15.66:0/2499349381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 msgr2=0x557dc0bbdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bbdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 -- 172.21.15.66:0/2499349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 msgr2=0x557dc0bc6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bc6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 -- 172.21.15.66:0/2499349381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc091bc60 con 0x557dc0bb53d0 2023-12-11T19:34:52.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f3824f9a700 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bc6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:52.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.826+0000 7f382579b700 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f381c0a3840 tx=0x7f381c003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb3d54f86251147 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f380bfff700 1 -- 172.21.15.66:0/2499349381 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c022470 con 0x557dc0bb53d0 2023-12-11T19:34:52.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f380bfff700 1 -- 172.21.15.66:0/2499349381 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557dc0ba8b40 con 0x557dc0bb53d0 2023-12-11T19:34:52.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f380bfff700 1 -- 172.21.15.66:0/2499349381 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c0a38d0 con 0x557dc0bb53d0 2023-12-11T19:34:52.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2499349381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 msgr2=0x557dc0bb57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f381c0a3840 tx=0x7f381c003e90 comp rx=0 tx=0).stop 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2499349381 shutdown_connections 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb6c20 0x557dc0bbdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bb57b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/2499349381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb5d30 0x557dc0bc6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.827+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2499349381 >> 172.21.15.66:0/2499349381 conn(0x557dc0ae47e0 msgr2=0x557dc0bbd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.828+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2499349381 shutdown_connections 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.828+0000 7f383b2c6b80 1 -- 172.21.15.66:0/2499349381 wait complete. 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.828+0000 7f383b2c6b80 1 -- start start 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.828+0000 7f383b2c6b80 1 Processor -- start 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bc4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bc4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 0x557dc0bc5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0bc5ea0 con 0x557dc0bb6c20 2023-12-11T19:34:52.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0bc6010 con 0x557dc0bb5d30 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc0bc61d0 con 0x557dc0bb53d0 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f383b2c6b80 10 client.0 _ll_register_callbacks cb 0x557dc09eea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f3825f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 0x557dc0bc5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f3825f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 0x557dc0bc5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44804/0 (socket says 172.21.15.66:44804) 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f3825f9c700 1 -- 172.21.15.66:0/4106238359 learned_addr learned my addr 172.21.15.66:0/4106238359 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f3824f9a700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bc4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.829+0000 7f382579b700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bc4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3825f9c700 1 -- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 msgr2=0x557dc0bc4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3825f9c700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bc4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3825f9c700 1 -- 172.21.15.66:0/4106238359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 msgr2=0x557dc0bc4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3825f9c700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bc4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3825f9c700 1 -- 172.21.15.66:0/4106238359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381002a040 con 0x557dc0bb6c20 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3825f9c700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 0x557dc0bc5960 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3818007bf0 tx=0x7f3818007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fc5fe5aa098570 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:52.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3808ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3808ff9700 1 -- 172.21.15.66:0/4106238359 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381800e040 con 0x557dc0bb6c20 2023-12-11T19:34:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3810021070 con 0x557dc0bb6c20 2023-12-11T19:34:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.830+0000 7f3808ff9700 1 -- 172.21.15.66:0/4106238359 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3818003e10 con 0x557dc0bb6c20 2023-12-11T19:34:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f3808ff9700 1 -- 172.21.15.66:0/4106238359 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3818004bb0 con 0x557dc0bb6c20 2023-12-11T19:34:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 10 client.15840 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:52.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557dc0932de0 con 0x557dc0bb6c20 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f3808ff9700 1 -- 172.21.15.66:0/4106238359 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3810021070 con 0x557dc0bb6c20 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f3808ff9700 1 -- 172.21.15.66:0/4106238359 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f381801e030 con 0x557dc0bb6c20 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 10 client.15840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 20 client.15840 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0c21f30 0x557dc0c24350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 4 client.15840 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.831+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557dc0acffd0 con 0x557dc0c21f30 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.832+0000 7f382579b700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0c21f30 0x557dc0c24350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.832+0000 7f382579b700 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0c21f30 0x557dc0c24350 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc7c7a7e6ead96b8 server_cookie=f251bfcbe0ca9faa in_seq=0 out_seq=0 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.832+0000 7f3808ff9700 10 client.15840 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:52.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f3808ff9700 1 -- 172.21.15.66:0/4106238359 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557dc0acffd0 con 0x557dc0c21f30 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f3808ff9700 10 client.15840 handle_command_reply: tid=0 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 client.15840 shutdown 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 msgr2=0x557dc0bc5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 0x557dc0bc5960 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3818007bf0 tx=0x7f3818007f00 comp rx=0 tx=0).stop 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 shutdown_connections 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x557dc0c21f30 0x557dc0c24350 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dc0bb6c20 0x557dc0bc5960 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557dc0bb53d0 0x557dc0bc4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f3808ff9700 0 client.15840 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 --2- 172.21.15.66:0/4106238359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557dc0bb5d30 0x557dc0bc4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 >> 172.21.15.66:0/4106238359 conn(0x557dc0ae47e0 msgr2=0x557dc0bbdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 shutdown_connections 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 1 -- 172.21.15.66:0/4106238359 wait complete. 2023-12-11T19:34:52.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.833+0000 7f383b2c6b80 20 client.15840 trim_cache size 0 max 16384 2023-12-11T19:34:52.838 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:52.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f380c0a1500 msgr2=0x7f380c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f380c0a1500 0x7f380c0a39c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3810009d20 tx=0x7f3810009260 comp rx=0 tx=0).stop 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201037b0 msgr2=0x7f3820199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201037b0 0x7f3820199720 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f381c0098e0 tx=0x7f381c00b9e0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 shutdown_connections 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f380c0a1500 0x7f380c0a39c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38201037b0 0x7f3820199720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3820105340 0x7f38201a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 --2- 172.21.15.66:0/327258710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38201049a0 0x7f3820199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 >> 172.21.15.66:0/327258710 conn(0x7f38200fed20 msgr2=0x7f3820107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 shutdown_connections 2023-12-11T19:34:52.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:52.834+0000 7f38279ff700 1 -- 172.21.15.66:0/327258710 wait complete. 2023-12-11T19:34:52.862 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-11T19:34:53.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:52 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2859456948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:34:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 -- 172.21.15.66:0/3045919145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8075610 msgr2=0x7f06d8073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 --2- 172.21.15.66:0/3045919145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8075610 0x7f06d8073a60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009990 tx=0x7f06d4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 -- 172.21.15.66:0/3045919145 shutdown_connections 2023-12-11T19:34:53.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 --2- 172.21.15.66:0/3045919145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8109460 0x7f06d810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 --2- 172.21.15.66:0/3045919145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8075610 0x7f06d8073a60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 --2- 172.21.15.66:0/3045919145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06d8074030 0x7f06d80744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 -- 172.21.15.66:0/3045919145 >> 172.21.15.66:0/3045919145 conn(0x7f06d80fcb10 msgr2=0x7f06d80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 -- 172.21.15.66:0/3045919145 shutdown_connections 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.076+0000 7f06dfdff700 1 -- 172.21.15.66:0/3045919145 wait complete. 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.077+0000 7f06dfdff700 1 Processor -- start 2023-12-11T19:34:53.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.077+0000 7f06dfdff700 1 -- start start 2023-12-11T19:34:53.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dfdff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06d8074030 0x7f06d8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dfdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8075610 0x7f06d8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dfdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 0x7f06d81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dfdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81a11b0 con 0x7f06d8109460 2023-12-11T19:34:53.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dfdff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81a12f0 con 0x7f06d8074030 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dfdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81a1480 con 0x7f06d8075610 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06de39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 0x7f06d81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06de39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 0x7f06d81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44834/0 (socket says 172.21.15.66:44834) 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06de39c700 1 -- 172.21.15.66:0/2967160790 learned_addr learned my addr 172.21.15.66:0/2967160790 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06d8074030 0x7f06d8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8075610 0x7f06d8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.078+0000 7f06de39c700 1 -- 172.21.15.66:0/2967160790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8075610 msgr2=0x7f06d8199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06de39c700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8075610 0x7f06d8199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06de39c700 1 -- 172.21.15.66:0/2967160790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06d8074030 msgr2=0x7f06d8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06de39c700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06d8074030 0x7f06d8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06de39c700 1 -- 172.21.15.66:0/2967160790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40095f0 con 0x7f06d8109460 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8075610 0x7f06d8199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06de39c700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 0x7f06d81a0be0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06d000ef30 tx=0x7f06d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06ceffd700 1 -- 172.21.15.66:0/2967160790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0018070 con 0x7f06d8109460 2023-12-11T19:34:53.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06ceffd700 1 -- 172.21.15.66:0/2967160790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06d000f040 con 0x7f06d8109460 2023-12-11T19:34:53.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06ceffd700 1 -- 172.21.15.66:0/2967160790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d000cc70 con 0x7f06d8109460 2023-12-11T19:34:53.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d81a1700 con 0x7f06d8109460 2023-12-11T19:34:53.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.079+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d81a1bc0 con 0x7f06d8109460 2023-12-11T19:34:53.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.083+0000 7f06ceffd700 1 -- 172.21.15.66:0/2967160790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f06d0007500 con 0x7f06d8109460 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.084+0000 7f06f37dab80 1 Processor -- start 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.084+0000 7f06ceffd700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06c40a1500 0x7f06c40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.084+0000 7f06f37dab80 1 -- start start 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.084+0000 7f06ceffd700 1 -- 172.21.15.66:0/2967160790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06d00cc840 con 0x7f06d8109460 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474520b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474520c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06c40a1500 0x7f06c40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06f37dab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564745165b50 con 0x56474520a3d0 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06f37dab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564745163470 con 0x56474520ad30 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06f37dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474515ed60 con 0x56474520bc20 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44846/0 (socket says 172.21.15.66:44846) 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 -- 172.21.15.66:0/883985466 learned_addr learned my addr 172.21.15.66:0/883985466 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06dd39a700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474520b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 -- 172.21.15.66:0/883985466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 msgr2=0x56474520c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06de39c700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474520c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474520c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 -- 172.21.15.66:0/883985466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 msgr2=0x56474520b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474520b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06ddb9b700 1 -- 172.21.15.66:0/883985466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564744fdb630 con 0x56474520a3d0 2023-12-11T19:34:53.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06dd39a700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474520b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:53.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.085+0000 7f06de39c700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474520c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:34:53.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.086+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06c40a1500 0x7f06c40a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f06d8075330 tx=0x7f06c80116c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.086+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520a7f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f06d4015000 tx=0x7f06d4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e3f00e75bb7cef server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.086+0000 7f06c3fff700 1 -- 172.21.15.66:0/883985466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401d030 con 0x56474520a3d0 2023-12-11T19:34:53.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.086+0000 7f06c3fff700 1 -- 172.21.15.66:0/883985466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06d401e040 con 0x56474520a3d0 2023-12-11T19:34:53.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.086+0000 7f06c3fff700 1 -- 172.21.15.66:0/883985466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400b7b0 con 0x56474520a3d0 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 -- 172.21.15.66:0/883985466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 msgr2=0x56474520a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520a7f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f06d4015000 tx=0x7f06d4005e80 comp rx=0 tx=0).stop 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 -- 172.21.15.66:0/883985466 shutdown_connections 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474520c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520a7f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 --2- 172.21.15.66:0/883985466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474520b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 -- 172.21.15.66:0/883985466 >> 172.21.15.66:0/883985466 conn(0x5647451397e0 msgr2=0x564745206d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 -- 172.21.15.66:0/883985466 shutdown_connections 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.087+0000 7f06f37dab80 1 -- 172.21.15.66:0/883985466 wait complete. 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.088+0000 7f06f37dab80 1 -- start start 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.088+0000 7f06f37dab80 1 Processor -- start 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x5647452140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520ad30 0x5647452145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520bc20 0x56474521b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564745118560 con 0x56474520a3d0 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564745118200 con 0x56474520bc20 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564744fdd920 con 0x56474520ad30 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06f37dab80 10 client.0 _ll_register_callbacks cb 0x564745043a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x5647452140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x5647452140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44854/0 (socket says 172.21.15.66:44854) 2023-12-11T19:34:53.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2832583889 learned_addr learned my addr 172.21.15.66:0/2832583889 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520ad30 0x5647452145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.089+0000 7f06de39c700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520bc20 0x56474521b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520ad30 msgr2=0x5647452145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520ad30 0x5647452145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2832583889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520bc20 msgr2=0x56474521b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520bc20 0x56474521b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2832583889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d810c510 con 0x56474520a3d0 2023-12-11T19:34:53.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520ad30 0x5647452145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x5647452140b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b2e0 tx=0x7f06d4003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d1cc2dde2306fb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06c17fa700 1 -- 172.21.15.66:0/2832583889 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401d040 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647451fdb40 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.090+0000 7f06c17fa700 1 -- 172.21.15.66:0/2832583889 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06d400be00 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06c17fa700 1 -- 172.21.15.66:0/2832583889 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d4023460 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 10 client.15852 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564744f87de0 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06c17fa700 1 -- 172.21.15.66:0/2832583889 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5647451fdb40 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06c17fa700 1 -- 172.21.15.66:0/2832583889 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06d400f620 con 0x56474520a3d0 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 10 client.15852 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 10 client.15852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 20 client.15852 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:53.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x56474521f9f0 0x564745221e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 4 client.15852 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:53.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.091+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647451d9f40 con 0x56474521f9f0 2023-12-11T19:34:53.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.092+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x56474521f9f0 0x564745221e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.092+0000 7f06c17fa700 10 client.15852 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:53.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.092+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x56474521f9f0 0x564745221e10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69da2d57be46969d server_cookie=39551d1da08182c8 in_seq=0 out_seq=0 2023-12-11T19:34:53.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06c17fa700 1 -- 172.21.15.66:0/2832583889 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647451d9f40 con 0x56474521f9f0 2023-12-11T19:34:53.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06c17fa700 10 client.15852 handle_command_reply: tid=0 2023-12-11T19:34:53.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 client.15852 shutdown 2023-12-11T19:34:53.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 msgr2=0x5647452140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x5647452140b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b2e0 tx=0x7f06d4003ea0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 shutdown_connections 2023-12-11T19:34:53.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x56474521f9f0 0x564745221e10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520bc20 0x56474521b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x5647452140b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06c17fa700 0 client.15852 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2832583889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520ad30 0x5647452145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.093+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 >> 172.21.15.66:0/2832583889 conn(0x5647451397e0 msgr2=0x564745205a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.094+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 shutdown_connections 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.094+0000 7f06f37dab80 1 -- 172.21.15.66:0/2832583889 wait complete. 2023-12-11T19:34:53.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.094+0000 7f06f37dab80 20 client.15852 trim_cache size 0 max 16384 2023-12-11T19:34:53.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.107+0000 7f06f37dab80 1 Processor -- start 2023-12-11T19:34:53.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 -- start start 2023-12-11T19:34:53.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474521b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x564745226770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474521c7f0 con 0x56474520a3d0 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564745201790 con 0x56474520ad30 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06f37dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474520ab70 con 0x56474520bc20 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44862/0 (socket says 172.21.15.66:44862) 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2187449805 learned_addr learned my addr 172.21.15.66:0/2187449805 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.108+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474521b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06de39c700 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x564745226770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.111 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:52 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2859456948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2187449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 msgr2=0x564745226770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x564745226770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2187449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 msgr2=0x56474521b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474521b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ddb9b700 1 -- 172.21.15.66:0/2187449805 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564744f70c60 con 0x56474520a3d0 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06dd39a700 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474521b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520c000 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f06d4000c00 tx=0x7f06d4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb18f035d827f92 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ccbff700 1 -- 172.21.15.66:0/2187449805 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400b7b0 con 0x56474520a3d0 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ccbff700 1 -- 172.21.15.66:0/2187449805 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06d400bc00 con 0x56474520a3d0 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.109+0000 7f06ccbff700 1 -- 172.21.15.66:0/2187449805 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40236a0 con 0x56474520a3d0 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 -- 172.21.15.66:0/2187449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 msgr2=0x56474520c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520c000 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f06d4000c00 tx=0x7f06d4001e70 comp rx=0 tx=0).stop 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 -- 172.21.15.66:0/2187449805 shutdown_connections 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x564745226770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x56474520c000 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 --2- 172.21.15.66:0/2187449805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x56474521b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 -- 172.21.15.66:0/2187449805 >> 172.21.15.66:0/2187449805 conn(0x5647451397e0 msgr2=0x564745213630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 -- 172.21.15.66:0/2187449805 shutdown_connections 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.110+0000 7f06f37dab80 1 -- 172.21.15.66:0/2187449805 wait complete. 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 -- start start 2023-12-11T19:34:53.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 Processor -- start 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x564745218cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x564745219230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474521a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474521a780 con 0x56474520a3d0 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474521a8f0 con 0x56474520ad30 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.111+0000 7f06f37dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474521aab0 con 0x56474520bc20 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06f37dab80 10 client.0 _ll_register_callbacks cb 0x564745043a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x564745218cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x564745218cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44870/0 (socket says 172.21.15.66:44870) 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 -- 172.21.15.66:0/817824186 learned_addr learned my addr 172.21.15.66:0/817824186 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06dd39a700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x564745219230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06de39c700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474521a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 -- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 msgr2=0x56474521a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474521a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 -- 172.21.15.66:0/817824186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 msgr2=0x564745219230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x564745219230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06ddb9b700 1 -- 172.21.15.66:0/817824186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c8011d20 con 0x56474520a3d0 2023-12-11T19:34:53.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.112+0000 7f06de39c700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474521a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06ddb9b700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x564745218cf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f06d4003b40 tx=0x7f06d4004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f16a3963241bef5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06c17fa700 1 -- 172.21.15.66:0/817824186 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401d030 con 0x56474520a3d0 2023-12-11T19:34:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06c17fa700 1 -- 172.21.15.66:0/817824186 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06d4023460 con 0x56474520a3d0 2023-12-11T19:34:53.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06d401e040 con 0x56474520a3d0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06c17fa700 1 -- 172.21.15.66:0/817824186 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d4023620 con 0x56474520a3d0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06f37dab80 10 client.15858 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.113+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564744f87de0 con 0x56474520a3d0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06c17fa700 1 -- 172.21.15.66:0/817824186 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06d401e040 con 0x56474520a3d0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06c17fa700 1 -- 172.21.15.66:0/817824186 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06d403a660 con 0x56474520a3d0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06f37dab80 10 client.15858 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06f37dab80 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06f37dab80 20 client.15858 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06f37dab80 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5647452763e0 0x564745278800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06f37dab80 4 client.15858 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564745124fd0 con 0x5647452763e0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.114+0000 7f06dd39a700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5647452763e0 0x564745278800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06dd39a700 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5647452763e0 0x564745278800 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12c3164dc0ceb172 server_cookie=65319e36a44f4f4c in_seq=0 out_seq=0 2023-12-11T19:34:53.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06c17fa700 10 client.15858 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06c17fa700 1 -- 172.21.15.66:0/817824186 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564745124fd0 con 0x5647452763e0 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06c17fa700 10 client.15858 handle_command_reply: tid=0 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06f37dab80 1 client.15858 shutdown 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 msgr2=0x564745218cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.115+0000 7f06f37dab80 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x564745218cf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f06d4003b40 tx=0x7f06d4004260 comp rx=0 tx=0).stop 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 shutdown_connections 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56474520bc20 0x56474521a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56474520a3d0 0x564745218cf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x5647452763e0 0x564745278800 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 --2- 172.21.15.66:0/817824186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56474520ad30 0x564745219230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 >> 172.21.15.66:0/817824186 conn(0x5647451397e0 msgr2=0x564745226770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06c17fa700 0 client.15858 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 shutdown_connections 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 1 -- 172.21.15.66:0/817824186 wait complete. 2023-12-11T19:34:53.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.116+0000 7f06f37dab80 20 client.15858 trim_cache size 0 max 16384 2023-12-11T19:34:53.121 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06c40a1500 msgr2=0x7f06c40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06c40a1500 0x7f06c40a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f06d8075330 tx=0x7f06c80116c0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 msgr2=0x7f06d81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 0x7f06d81a0be0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06d000ef30 tx=0x7f06d000c3a0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 shutdown_connections 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f06c40a1500 0x7f06c40a39c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06d8074030 0x7f06d8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06d8109460 0x7f06d81a0be0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 --2- 172.21.15.66:0/2967160790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06d8075610 0x7f06d8199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 >> 172.21.15.66:0/2967160790 conn(0x7f06d80fcb10 msgr2=0x7f06d8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 shutdown_connections 2023-12-11T19:34:53.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.117+0000 7f06dfdff700 1 -- 172.21.15.66:0/2967160790 wait complete. 2023-12-11T19:34:53.137 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-11T19:34:53.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 -- 172.21.15.66:0/3932757578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc102790 msgr2=0x7f0abc102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/3932757578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc102790 0x7f0abc102bf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00099c0 tx=0x7f0ab0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 -- 172.21.15.66:0/3932757578 shutdown_connections 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/3932757578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc1015a0 0x7f0abc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/3932757578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc103130 0x7f0abc1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/3932757578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc102790 0x7f0abc102bf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.348+0000 7f0ac25ff700 1 -- 172.21.15.66:0/3932757578 >> 172.21.15.66:0/3932757578 conn(0x7f0abc0fcb10 msgr2=0x7f0abc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.349+0000 7f0ac25ff700 1 -- 172.21.15.66:0/3932757578 shutdown_connections 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.349+0000 7f0ac25ff700 1 -- 172.21.15.66:0/3932757578 wait complete. 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 Processor -- start 2023-12-11T19:34:53.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 -- start start 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 0x7f0abc197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc102790 0x7f0abc1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc103130 0x7f0abc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc19efa0 con 0x7f0abc103130 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc19f0e0 con 0x7f0abc102790 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac25ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc19f270 con 0x7f0abc1015a0 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0ac0b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc103130 0x7f0abc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0abbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 0x7f0abc197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0abbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 0x7f0abc197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:40890/0 (socket says 172.21.15.66:40890) 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.350+0000 7f0abbfff700 1 -- 172.21.15.66:0/1181999193 learned_addr learned my addr 172.21.15.66:0/1181999193 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc102790 0x7f0abc1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abbfff700 1 -- 172.21.15.66:0/1181999193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc102790 msgr2=0x7f0abc1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc102790 0x7f0abc1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abbfff700 1 -- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc103130 msgr2=0x7f0abc19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc103130 0x7f0abc19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abbfff700 1 -- 172.21.15.66:0/1181999193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00095f0 con 0x7f0abc1015a0 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 0x7f0abc197470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00e970 tx=0x7f0aac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0ab97fa700 1 -- 172.21.15.66:0/1181999193 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac009790 con 0x7f0abc1015a0 2023-12-11T19:34:53.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0ab97fa700 1 -- 172.21.15.66:0/1181999193 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aac00c7e0 con 0x7f0abc1015a0 2023-12-11T19:34:53.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc19f550 con 0x7f0abc1015a0 2023-12-11T19:34:53.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.351+0000 7f0ab97fa700 1 -- 172.21.15.66:0/1181999193 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003bd0 con 0x7f0abc1015a0 2023-12-11T19:34:53.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.352+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc19fa60 con 0x7f0abc1015a0 2023-12-11T19:34:53.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.357+0000 7f0ab97fa700 1 -- 172.21.15.66:0/1181999193 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0aac00c950 con 0x7f0abc1015a0 2023-12-11T19:34:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.357+0000 7f0ad5ee6b80 1 Processor -- start 2023-12-11T19:34:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.357+0000 7f0ad5ee6b80 1 -- start start 2023-12-11T19:34:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506725ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506725fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650671b5b50 con 0x56506725f140 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650671b3470 con 0x565067260030 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650671aed60 con 0x56506725e7e0 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0abbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506725fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0abbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506725fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44892/0 (socket says 172.21.15.66:44892) 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0abbfff700 1 -- 172.21.15.66:0/3663344874 learned_addr learned my addr 172.21.15.66:0/3663344874 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506725ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.358+0000 7f0ab97fa700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0aa40a1480 0x7f0aa40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0abbfff700 1 -- 172.21.15.66:0/3663344874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 msgr2=0x56506725ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0abbfff700 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506725ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0abbfff700 1 -- 172.21.15.66:0/3663344874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 msgr2=0x5650672604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0abbfff700 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0abbfff700 1 -- 172.21.15.66:0/3663344874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56506702b630 con 0x56506725f140 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0ab97fa700 1 -- 172.21.15.66:0/1181999193 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0aac014070 con 0x7f0abc1015a0 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0aa40a1480 0x7f0aa40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0abbfff700 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506725fa50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0aac01fbc0 tx=0x7f0aac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f5bee1f1ba3b2f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0aa27fc700 1 -- 172.21.15.66:0/3663344874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac099d10 con 0x56506725f140 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0aa27fc700 1 -- 172.21.15.66:0/3663344874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aac099b30 con 0x56506725f140 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.359+0000 7f0aa27fc700 1 -- 172.21.15.66:0/3663344874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00cc50 con 0x56506725f140 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0aa40a1480 0x7f0aa40a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009980 tx=0x7f0ab4004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/3663344874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 msgr2=0x56506725fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506725fa50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0aac01fbc0 tx=0x7f0aac020000 comp rx=0 tx=0).stop 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/3663344874 shutdown_connections 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506725ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/3663344874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506725fa50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/3663344874 >> 172.21.15.66:0/3663344874 conn(0x5650671897e0 msgr2=0x565067256d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/3663344874 shutdown_connections 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.360+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/3663344874 wait complete. 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 -- start start 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 Processor -- start 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56506725e7e0 0x5650672684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725f140 0x565067268a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 0x56506726fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x565067168560 con 0x565067260030 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x565067168200 con 0x56506725e7e0 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506702d920 con 0x56506725f140 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.361+0000 7f0ad5ee6b80 10 client.0 _ll_register_callbacks cb 0x565067093a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 0x56506726fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 0x56506726fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44906/0 (socket says 172.21.15.66:44906) 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0abbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56506725e7e0 0x5650672684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/1161909234 learned_addr learned my addr 172.21.15.66:0/1161909234 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725f140 0x565067268a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/1161909234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725f140 msgr2=0x565067268a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725f140 0x565067268a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56506725e7e0 msgr2=0x5650672684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56506725e7e0 0x5650672684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/1161909234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc19fa60 con 0x565067260030 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56506725e7e0 0x5650672684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 0x56506726fa70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4008ac0 tx=0x7f0ab4009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3822e0cc131e1bd9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.362+0000 7f0a9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:53.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1161909234 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4017cf0 con 0x565067260030 2023-12-11T19:34:53.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56506724db40 con 0x565067260030 2023-12-11T19:34:53.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1161909234 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0ab401c030 con 0x565067260030 2023-12-11T19:34:53.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1161909234 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab4017e50 con 0x565067260030 2023-12-11T19:34:53.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0ad5ee6b80 10 client.15870 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:53.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565066fd7de0 con 0x565067260030 2023-12-11T19:34:53.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1161909234 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab4032020 con 0x565067260030 2023-12-11T19:34:53.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1161909234 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ab402c400 con 0x565067260030 2023-12-11T19:34:53.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0ad5ee6b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:53.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.363+0000 7f0ad5ee6b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:53.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.364+0000 7f0ad5ee6b80 20 client.15870 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:53.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.364+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672743b0 0x5650672767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.364+0000 7f0ad5ee6b80 4 client.15870 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.364+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565067229f40 con 0x5650672743b0 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.364+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672743b0 0x5650672767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.365+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672743b0 0x5650672767d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b5567cd3af6993 server_cookie=2a23222c28a5133d in_seq=0 out_seq=0 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.365+0000 7f0a9b7fe700 10 client.15870 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.365+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1161909234 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565067229f40 con 0x5650672743b0 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.365+0000 7f0a9b7fe700 10 client.15870 handle_command_reply: tid=0 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.365+0000 7f0ad5ee6b80 1 client.15870 shutdown 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 msgr2=0x56506726fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 0x56506726fa70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4008ac0 tx=0x7f0ab4009100 comp rx=0 tx=0).stop 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 shutdown_connections 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672743b0 0x5650672767d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56506725e7e0 0x5650672684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0a9b7fe700 0 client.15870 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:53.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x565067260030 0x56506726fa70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1161909234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725f140 0x565067268a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 >> 172.21.15.66:0/1161909234 conn(0x5650671897e0 msgr2=0x565067255a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 shutdown_connections 2023-12-11T19:34:53.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1161909234 wait complete. 2023-12-11T19:34:53.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.366+0000 7f0ad5ee6b80 20 client.15870 trim_cache size 0 max 16384 2023-12-11T19:34:53.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 Processor -- start 2023-12-11T19:34:53.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 -- start start 2023-12-11T19:34:53.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x565067260410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x565067251b50 con 0x56506725f140 2023-12-11T19:34:53.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x565067266fc0 con 0x565067260030 2023-12-11T19:34:53.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506727b9c0 con 0x56506725e7e0 2023-12-11T19:34:53.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.380+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44922/0 (socket says 172.21.15.66:44922) 2023-12-11T19:34:53.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0abb7fe700 1 -- 172.21.15.66:0/681839709 learned_addr learned my addr 172.21.15.66:0/681839709 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0abbfff700 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x565067260410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/681839709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 msgr2=0x565067260410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x565067260410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/681839709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 msgr2=0x56506726fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 -- 172.21.15.66:0/681839709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565066fc0c60 con 0x565067260030 2023-12-11T19:34:53.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672664f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402c780 tx=0x7f0ab4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6d4ea266d975f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.381+0000 7f0aa2ffd700 1 -- 172.21.15.66:0/681839709 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4008e80 con 0x565067260030 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0aa2ffd700 1 -- 172.21.15.66:0/681839709 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab4032070 con 0x565067260030 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0aa2ffd700 1 -- 172.21.15.66:0/681839709 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4039450 con 0x565067260030 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/681839709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 msgr2=0x5650672664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672664f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402c780 tx=0x7f0ab4000f40 comp rx=0 tx=0).stop 2023-12-11T19:34:53.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/681839709 shutdown_connections 2023-12-11T19:34:53.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x5650672664f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x565067260410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/681839709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/681839709 >> 172.21.15.66:0/681839709 conn(0x5650671897e0 msgr2=0x565067267a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/681839709 shutdown_connections 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.382+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/681839709 wait complete. 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 -- start start 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 Processor -- start 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506726b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x56506726e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506726eeb0 con 0x56506725f140 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506726f020 con 0x565067260030 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56506726f1e0 con 0x56506725e7e0 2023-12-11T19:34:53.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.383+0000 7f0ad5ee6b80 10 client.0 _ll_register_callbacks cb 0x565067093a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44928/0 (socket says 172.21.15.66:44928) 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 -- 172.21.15.66:0/1058928600 learned_addr learned my addr 172.21.15.66:0/1058928600 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0ac0b9c700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x56506726e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506726b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 -- 172.21.15.66:0/1058928600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 msgr2=0x56506726b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506726b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 -- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 msgr2=0x56506726e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x56506726e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 -- 172.21.15.66:0/1058928600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc10a330 con 0x56506725f140 2023-12-11T19:34:53.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0abb7fe700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726b9a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009990 tx=0x7f0ab00052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b97065841e883a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.384+0000 7f0a9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1058928600 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000bd10 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aac099b30 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1058928600 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0ab0003760 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1058928600 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab0003920 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 10 client.15876 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565066fd7de0 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1058928600 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aac099b30 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1058928600 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ab0023040 con 0x56506725f140 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 20 client.15876 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672cb4c0 0x5650672cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:53.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 4 client.15876 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.385+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565067174fd0 con 0x5650672cb4c0 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.386+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672cb4c0 0x5650672cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.386+0000 7f0abbfff700 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672cb4c0 0x5650672cd8e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d7b195b0396f6b server_cookie=c004e87a04657b in_seq=0 out_seq=0 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.386+0000 7f0a9b7fe700 10 client.15876 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0a9b7fe700 1 -- 172.21.15.66:0/1058928600 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565067174fd0 con 0x5650672cb4c0 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0a9b7fe700 10 client.15876 handle_command_reply: tid=0 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 client.15876 shutdown 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 msgr2=0x56506726b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726b9a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009990 tx=0x7f0ab00052e0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 shutdown_connections 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5650672cb4c0 0x5650672cd8e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x565067260030 0x56506726e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56506725e7e0 0x56506726b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0a9b7fe700 0 client.15876 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 --2- 172.21.15.66:0/1058928600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56506725f140 0x56506726b9a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 >> 172.21.15.66:0/1058928600 conn(0x5650671897e0 msgr2=0x565067266470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 shutdown_connections 2023-12-11T19:34:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 1 -- 172.21.15.66:0/1058928600 wait complete. 2023-12-11T19:34:53.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.387+0000 7f0ad5ee6b80 20 client.15876 trim_cache size 0 max 16384 2023-12-11T19:34:53.403 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0aa40a1480 msgr2=0x7f0aa40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0aa40a1480 0x7f0aa40a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009980 tx=0x7f0ab4004ad0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 msgr2=0x7f0abc197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 0x7f0abc197470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00e970 tx=0x7f0aac00ec80 comp rx=0 tx=0).stop 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 shutdown_connections 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f0aa40a1480 0x7f0aa40a3940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0abc103130 0x7f0abc19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0abc1015a0 0x7f0abc197470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 --2- 172.21.15.66:0/1181999193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc102790 0x7f0abc1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:53.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 >> 172.21.15.66:0/1181999193 conn(0x7f0abc0fcb10 msgr2=0x7f0abc105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:53.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 shutdown_connections 2023-12-11T19:34:53.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:53.388+0000 7f0ac25ff700 1 -- 172.21.15.66:0/1181999193 wait complete. 2023-12-11T19:34:54.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:53 smithi167 ceph-mon[102750]: pgmap v399: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2023-12-11T19:34:54.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:53 smithi066 ceph-mon[97920]: pgmap v399: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2023-12-11T19:34:54.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:53 smithi070 ceph-mon[104044]: pgmap v399: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2023-12-11T19:34:54.409 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-11T19:34:54.410 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-11T19:34:54.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 -- 172.21.15.66:0/853079878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4102000 msgr2=0x7f08d41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 --2- 172.21.15.66:0/853079878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4102000 0x7f08d41043f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009990 tx=0x7f08c4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 -- 172.21.15.66:0/853079878 shutdown_connections 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 --2- 172.21.15.66:0/853079878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4107260 0x7f08d41096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 --2- 172.21.15.66:0/853079878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4102000 0x7f08d41043f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 --2- 172.21.15.66:0/853079878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d4104930 0x7f08d4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 -- 172.21.15.66:0/853079878 >> 172.21.15.66:0/853079878 conn(0x7f08d40fb9f0 msgr2=0x7f08d40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 -- 172.21.15.66:0/853079878 shutdown_connections 2023-12-11T19:34:54.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.624+0000 7f08da3ff700 1 -- 172.21.15.66:0/853079878 wait complete. 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.625+0000 7f08da3ff700 1 Processor -- start 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.625+0000 7f08da3ff700 1 -- start start 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08da3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4102000 0x7f08d4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08da3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 0x7f08d4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08da3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d4107260 0x7f08d419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08da3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419ef40 con 0x7f08d4104930 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08da3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419f080 con 0x7f08d4102000 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08da3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419f210 con 0x7f08d4107260 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 0x7f08d4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 0x7f08d4197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44952/0 (socket says 172.21.15.66:44952) 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08d37fe700 1 -- 172.21.15.66:0/3356686589 learned_addr learned my addr 172.21.15.66:0/3356686589 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4102000 0x7f08d4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.626+0000 7f08d899c700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d4107260 0x7f08d419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d37fe700 1 -- 172.21.15.66:0/3356686589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d4107260 msgr2=0x7f08d419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d4107260 0x7f08d419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d37fe700 1 -- 172.21.15.66:0/3356686589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4102000 msgr2=0x7f08d4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4102000 0x7f08d4197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d37fe700 1 -- 172.21.15.66:0/3356686589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c40095f0 con 0x7f08d4104930 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4102000 0x7f08d4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:54.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 0x7f08d4197950 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f08c800d950 tx=0x7f08c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d17fa700 1 -- 172.21.15.66:0/3356686589 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800f890 con 0x7f08d4104930 2023-12-11T19:34:54.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d17fa700 1 -- 172.21.15.66:0/3356686589 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c800fe80 con 0x7f08d4104930 2023-12-11T19:34:54.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08d17fa700 1 -- 172.21.15.66:0/3356686589 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800e3b0 con 0x7f08d4104930 2023-12-11T19:34:54.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.627+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d419f4f0 con 0x7f08d4104930 2023-12-11T19:34:54.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.628+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d419fa00 con 0x7f08d4104930 2023-12-11T19:34:54.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.631+0000 7f08d17fa700 1 -- 172.21.15.66:0/3356686589 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f08c8010460 con 0x7f08d4104930 2023-12-11T19:34:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.633+0000 7f08edc4ab80 1 Processor -- start 2023-12-11T19:34:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.633+0000 7f08edc4ab80 1 -- start start 2023-12-11T19:34:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d17fa700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f08bc0a1500 0x7f08bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 0x564986294870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986295ca0 0x564986296120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649861efbd0 con 0x564986295ca0 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649861ed4f0 con 0x564986294450 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649861e8de0 con 0x564986294db0 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d17fa700 1 -- 172.21.15.66:0/3356686589 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08c80cca80 con 0x7f08d4104930 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d899c700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f08bc0a1500 0x7f08bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 0x564986294870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 0x564986294870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:56342/0 (socket says 172.21.15.66:56342) 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d3fff700 1 -- 172.21.15.66:0/3694033303 learned_addr learned my addr 172.21.15.66:0/3694033303 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d899c700 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986295ca0 0x564986296120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.634+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d3fff700 1 -- 172.21.15.66:0/3694033303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 msgr2=0x5649862956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d3fff700 1 -- 172.21.15.66:0/3694033303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986295ca0 msgr2=0x564986296120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986295ca0 0x564986296120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d3fff700 1 -- 172.21.15.66:0/3694033303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649860656b0 con 0x564986294450 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 0x564986294870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08c4000c00 tx=0x7f08c4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3d923a75a0940e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08d899c700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f08bc0a1500 0x7f08bc0a39c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x564986295480 tx=0x7f08cc009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08ba7fc700 1 -- 172.21.15.66:0/3694033303 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c401d070 con 0x564986294450 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08ba7fc700 1 -- 172.21.15.66:0/3694033303 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c401e040 con 0x564986294450 2023-12-11T19:34:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.635+0000 7f08ba7fc700 1 -- 172.21.15.66:0/3694033303 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400bba0 con 0x564986294450 2023-12-11T19:34:54.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3694033303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 msgr2=0x564986294870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 0x564986294870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08c4000c00 tx=0x7f08c4005c00 comp rx=0 tx=0).stop 2023-12-11T19:34:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3694033303 shutdown_connections 2023-12-11T19:34:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294450 0x564986294870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986295ca0 0x564986296120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3694033303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3694033303 >> 172.21.15.66:0/3694033303 conn(0x5649861c3860 msgr2=0x564986290e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3694033303 shutdown_connections 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.636+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3694033303 wait complete. 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.637+0000 7f08edc4ab80 1 -- start start 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.637+0000 7f08edc4ab80 1 Processor -- start 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294450 0x56498629e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 0x56498629e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x5649862a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649861a25e0 con 0x564986294db0 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649861a2280 con 0x564986295ca0 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649860679a0 con 0x564986294450 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08edc4ab80 10 client.0 _ll_register_callbacks cb 0x5649860cdb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 0x56498629e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 0x56498629e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44970/0 (socket says 172.21.15.66:44970) 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 -- 172.21.15.66:0/3190565301 learned_addr learned my addr 172.21.15.66:0/3190565301 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294450 0x56498629e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d899c700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x5649862a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 -- 172.21.15.66:0/3190565301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294450 msgr2=0x56498629e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294450 0x56498629e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 -- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 msgr2=0x5649862a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x5649862a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d37fe700 1 -- 172.21.15.66:0/3190565301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d410a2c0 con 0x564986294db0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.638+0000 7f08d899c700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x5649862a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08d37fe700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 0x56498629e730 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f08c8096260 tx=0x7f08c8010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724397431df2bfd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08b37fe700 1 -- 172.21.15.66:0/3190565301 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c809deb0 con 0x564986294db0 2023-12-11T19:34:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564986287bc0 con 0x564986294db0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08b37fe700 1 -- 172.21.15.66:0/3190565301 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08c80a20f0 con 0x564986294db0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08b37fe700 1 -- 172.21.15.66:0/3190565301 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c8099fc0 con 0x564986294db0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08edc4ab80 10 client.15888 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564986011e60 con 0x564986294db0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.639+0000 7f08b37fe700 1 -- 172.21.15.66:0/3190565301 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c80b1080 con 0x564986294db0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08b37fe700 1 -- 172.21.15.66:0/3190565301 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f08c80a1c60 con 0x564986294db0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08edc4ab80 10 client.15888 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08edc4ab80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08edc4ab80 20 client.15888 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5649862ae380 0x5649862b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08edc4ab80 4 client.15888 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564986263fc0 con 0x5649862ae380 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.640+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5649862ae380 0x5649862b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.641+0000 7f08d3fff700 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5649862ae380 0x5649862b07a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6984eee9887396 server_cookie=58d4f64a2798b2c1 in_seq=0 out_seq=0 2023-12-11T19:34:54.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.641+0000 7f08b37fe700 10 client.15888 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08b37fe700 1 -- 172.21.15.66:0/3190565301 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564986263fc0 con 0x5649862ae380 2023-12-11T19:34:54.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08b37fe700 10 client.15888 handle_command_reply: tid=0 2023-12-11T19:34:54.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 client.15888 shutdown 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 msgr2=0x56498629e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 0x56498629e730 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f08c8096260 tx=0x7f08c8010850 comp rx=0 tx=0).stop 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 shutdown_connections 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5649862ae380 0x5649862b07a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x5649862a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294450 0x56498629e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/3190565301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294db0 0x56498629e730 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 >> 172.21.15.66:0/3190565301 conn(0x5649861c3860 msgr2=0x56498628fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08b37fe700 0 client.15888 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 shutdown_connections 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.642+0000 7f08edc4ab80 1 -- 172.21.15.66:0/3190565301 wait complete. 2023-12-11T19:34:54.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.643+0000 7f08edc4ab80 20 client.15888 trim_cache size 0 max 16384 2023-12-11T19:34:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.660+0000 7f08edc4ab80 1 Processor -- start 2023-12-11T19:34:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 -- start start 2023-12-11T19:34:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x564986290f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564986295190 con 0x564986294450 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498629cc30 con 0x564986295ca0 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498629f3e0 con 0x564986294db0 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862909f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44980/0 (socket says 172.21.15.66:44980) 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08d3fff700 1 -- 172.21.15.66:0/1959188722 learned_addr learned my addr 172.21.15.66:0/1959188722 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08d37fe700 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.661+0000 7f08d899c700 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x564986290f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d3fff700 1 -- 172.21.15.66:0/1959188722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 msgr2=0x5649862a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d3fff700 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d3fff700 1 -- 172.21.15.66:0/1959188722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 msgr2=0x564986290f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d3fff700 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x564986290f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d3fff700 1 -- 172.21.15.66:0/1959188722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564985fface0 con 0x564986294450 2023-12-11T19:34:54.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d899c700 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x564986290f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:54.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08d3fff700 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862909f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f08c4003f60 tx=0x7f08c4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7b8267f970b305 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08baffd700 1 -- 172.21.15.66:0/1959188722 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c401d040 con 0x564986294450 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08baffd700 1 -- 172.21.15.66:0/1959188722 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c400b780 con 0x564986294450 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.662+0000 7f08baffd700 1 -- 172.21.15.66:0/1959188722 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4032760 con 0x564986294450 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 -- 172.21.15.66:0/1959188722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 msgr2=0x5649862909f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862909f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f08c4003f60 tx=0x7f08c4009ca0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 -- 172.21.15.66:0/1959188722 shutdown_connections 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986295ca0 0x564986290f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862909f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/1959188722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986294db0 0x5649862a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 -- 172.21.15.66:0/1959188722 >> 172.21.15.66:0/1959188722 conn(0x5649861c3860 msgr2=0x564986298bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 -- 172.21.15.66:0/1959188722 shutdown_connections 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.663+0000 7f08edc4ab80 1 -- 172.21.15.66:0/1959188722 wait complete. 2023-12-11T19:34:54.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 -- start start 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 Processor -- start 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294db0 0x5649862a2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986295ca0 0x5649862a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564986304870 con 0x564986294450 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649863049b0 con 0x564986294db0 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.664+0000 7f08edc4ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564986304af0 con 0x564986295ca0 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08edc4ab80 10 client.0 _ll_register_callbacks cb 0x5649860cdb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44988/0 (socket says 172.21.15.66:44988) 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 -- 172.21.15.66:0/2038414836 learned_addr learned my addr 172.21.15.66:0/2038414836 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d899c700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986295ca0 0x5649862a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d37fe700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294db0 0x5649862a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 -- 172.21.15.66:0/2038414836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986295ca0 msgr2=0x5649862a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986295ca0 0x5649862a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 -- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294db0 msgr2=0x5649862a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294db0 0x5649862a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 -- 172.21.15.66:0/2038414836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649860656b0 con 0x564986294450 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08d3fff700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862a5640 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f08c4003b80 tx=0x7f08c4003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58550ffd0fc6fab server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.665+0000 7f08b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08b37fe700 1 -- 172.21.15.66:0/2038414836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c401d040 con 0x564986294450 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08b37fe700 1 -- 172.21.15.66:0/2038414836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08c400be80 con 0x564986294450 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56498629d560 con 0x564986294450 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08b37fe700 1 -- 172.21.15.66:0/2038414836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c4005040 con 0x564986294450 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08edc4ab80 10 client.15894 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:54.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5649860382d0 con 0x564986294450 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.666+0000 7f08b37fe700 1 -- 172.21.15.66:0/2038414836 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56498629d560 con 0x564986294450 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08b37fe700 1 -- 172.21.15.66:0/2038414836 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f08c403a030 con 0x564986294450 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08edc4ab80 10 client.15894 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08edc4ab80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08edc4ab80 20 client.15894 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x564986306090 0x5649863084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08edc4ab80 4 client.15894 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f08c401e040 con 0x564986306090 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.667+0000 7f08d37fe700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x564986306090 0x5649863084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.668+0000 7f08b37fe700 10 client.15894 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.668+0000 7f08d37fe700 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x564986306090 0x5649863084b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733e20ff9ada06cf server_cookie=65ceddfc4965cf84 in_seq=0 out_seq=0 2023-12-11T19:34:54.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08b37fe700 1 -- 172.21.15.66:0/2038414836 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f08c401e040 con 0x564986306090 2023-12-11T19:34:54.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08b37fe700 10 client.15894 handle_command_reply: tid=0 2023-12-11T19:34:54.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 client.15894 shutdown 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 msgr2=0x5649862a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862a5640 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f08c4003b80 tx=0x7f08c4003de0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 shutdown_connections 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x564986306090 0x5649863084b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564986295ca0 0x5649862a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564986294450 0x5649862a5640 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08b37fe700 0 client.15894 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 --2- 172.21.15.66:0/2038414836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564986294db0 0x5649862a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 >> 172.21.15.66:0/2038414836 conn(0x5649861c3860 msgr2=0x5649862909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 shutdown_connections 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.669+0000 7f08edc4ab80 1 -- 172.21.15.66:0/2038414836 wait complete. 2023-12-11T19:34:54.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08edc4ab80 20 client.15894 trim_cache size 0 max 16384 2023-12-11T19:34:54.674 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:34:54.674 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-12-11T19:34:54.674 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "bb753821-e915-494e-a5bb-e5ddd58168b3", 2023-12-11T19:34:54.674 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-12-11T19:34:54.674 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:34:54.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f08bc0a1500 msgr2=0x7f08bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f08bc0a1500 0x7f08bc0a39c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x564986295480 tx=0x7f08cc009920 comp rx=0 tx=0).stop 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 msgr2=0x7f08d4197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 0x7f08d4197950 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f08c800d950 tx=0x7f08c800dc60 comp rx=0 tx=0).stop 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 shutdown_connections 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f08bc0a1500 0x7f08bc0a39c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08d4102000 0x7f08d4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d4107260 0x7f08d419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 --2- 172.21.15.66:0/3356686589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08d4104930 0x7f08d4197950 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 >> 172.21.15.66:0/3356686589 conn(0x7f08d40fb9f0 msgr2=0x7f08d4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 shutdown_connections 2023-12-11T19:34:54.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.670+0000 7f08da3ff700 1 -- 172.21.15.66:0/3356686589 wait complete. 2023-12-11T19:34:54.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:34:54.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3289614296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 msgr2=0x7ffb281019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3289614296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb281019c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffb180099c0 tx=0x7ffb18009cd0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3289614296 shutdown_connections 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3289614296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28103130 0x7ffb281099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3289614296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb281019c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3289614296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb28102790 0x7ffb28102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.903+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3289614296 >> 172.21.15.66:0/3289614296 conn(0x7ffb280fcb10 msgr2=0x7ffb280fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.904+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3289614296 shutdown_connections 2023-12-11T19:34:54.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.904+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3289614296 wait complete. 2023-12-11T19:34:54.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.904+0000 7ffb2dbff700 1 Processor -- start 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 -- start start 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb28075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28102790 0x7ffb280739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb28103130 0x7ffb28073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2819b310 con 0x7ffb281015a0 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28074440 con 0x7ffb28103130 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb2dbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28074600 con 0x7ffb28102790 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb28075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb28075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44994/0 (socket says 172.21.15.66:44994) 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb277fe700 1 -- 172.21.15.66:0/3109541071 learned_addr learned my addr 172.21.15.66:0/3109541071 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb27fff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb28103130 0x7ffb28073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.905+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28102790 0x7ffb280739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb277fe700 1 -- 172.21.15.66:0/3109541071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28102790 msgr2=0x7ffb280739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb277fe700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28102790 0x7ffb280739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb277fe700 1 -- 172.21.15.66:0/3109541071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb28103130 msgr2=0x7ffb28073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb277fe700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb28103130 0x7ffb28073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb277fe700 1 -- 172.21.15.66:0/3109541071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb180095f0 con 0x7ffb281015a0 2023-12-11T19:34:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28102790 0x7ffb280739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:34:54.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb277fe700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb28075330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009fd0 tx=0x7ffb1800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb24ff9700 1 -- 172.21.15.66:0/3109541071 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1801d070 con 0x7ffb281015a0 2023-12-11T19:34:54.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb28074820 con 0x7ffb281015a0 2023-12-11T19:34:54.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb24ff9700 1 -- 172.21.15.66:0/3109541071 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb1800bd10 con 0x7ffb281015a0 2023-12-11T19:34:54.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb24ff9700 1 -- 172.21.15.66:0/3109541071 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18004ba0 con 0x7ffb281015a0 2023-12-11T19:34:54.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.906+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb281a5f60 con 0x7ffb281015a0 2023-12-11T19:34:54.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.910+0000 7ffb24ff9700 1 -- 172.21.15.66:0/3109541071 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ffb18004d00 con 0x7ffb281015a0 2023-12-11T19:34:54.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.911+0000 7ffb24ff9700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ffb100a1480 0x7ffb100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.911+0000 7ffb24ff9700 1 -- 172.21.15.66:0/3109541071 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb180cdb40 con 0x7ffb281015a0 2023-12-11T19:34:54.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.912+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ffb100a1480 0x7ffb100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.912+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ffb100a1480 0x7ffb100a3940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009cb0 tx=0x7ffb1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.913+0000 7ffb4159eb80 1 Processor -- start 2023-12-11T19:34:54.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 -- start start 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba30db50 con 0x55f0ba3b3c20 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba30b470 con 0x55f0ba3b23d0 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba306d60 con 0x55f0ba3b2d30 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45002/0 (socket says 172.21.15.66:45002) 2023-12-11T19:34:54.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.914+0000 7ffb27fff700 1 -- 172.21.15.66:0/4247013934 learned_addr learned my addr 172.21.15.66:0/4247013934 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb277fe700 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb27fff700 1 -- 172.21.15.66:0/4247013934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 msgr2=0x55f0ba3b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb27fff700 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb27fff700 1 -- 172.21.15.66:0/4247013934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b23d0 msgr2=0x55f0ba3b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb27fff700 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb27fff700 1 -- 172.21.15.66:0/4247013934 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0ba183630 con 0x55f0ba3b3c20 2023-12-11T19:34:54.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb27fff700 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3b40a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000f5a0 tx=0x7ffb2000f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503b437c84ab9062 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb0dffb700 1 -- 172.21.15.66:0/4247013934 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20004bb0 con 0x55f0ba3b3c20 2023-12-11T19:34:54.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb0dffb700 1 -- 172.21.15.66:0/4247013934 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb20004d10 con 0x55f0ba3b3c20 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.915+0000 7ffb0dffb700 1 -- 172.21.15.66:0/4247013934 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20005020 con 0x55f0ba3b3c20 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.916+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4247013934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 msgr2=0x55f0ba3b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.916+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3b40a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000f5a0 tx=0x7ffb2000f8b0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4247013934 shutdown_connections 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3b40a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4247013934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4247013934 >> 172.21.15.66:0/4247013934 conn(0x55f0ba2e17e0 msgr2=0x55f0ba3aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4247013934 shutdown_connections 2023-12-11T19:34:54.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4247013934 wait complete. 2023-12-11T19:34:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 -- start start 2023-12-11T19:34:54.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.917+0000 7ffb4159eb80 1 Processor -- start 2023-12-11T19:34:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba2c0560 con 0x55f0ba3b23d0 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba2c0200 con 0x55f0ba3b3c20 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba185920 con 0x55f0ba3b2d30 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb4159eb80 10 client.0 _ll_register_callbacks cb 0x55f0ba1eba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45010/0 (socket says 172.21.15.66:45010) 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:56424/0 (socket says 172.21.15.66:56424) 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb277fe700 1 -- 172.21.15.66:0/4127512978 learned_addr learned my addr 172.21.15.66:0/4127512978 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.918+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb27fff700 1 -- 172.21.15.66:0/4127512978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 msgr2=0x55f0ba3bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb27fff700 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb27fff700 1 -- 172.21.15.66:0/4127512978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 msgr2=0x55f0ba3bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb27fff700 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb27fff700 1 -- 172.21.15.66:0/4127512978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2000f1d0 con 0x55f0ba3b3c20 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb27fff700 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c3760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000f570 tx=0x7ffb20003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224eb6803224e77d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.919+0000 7ffb06ffd700 1 -- 172.21.15.66:0/4127512978 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20014680 con 0x55f0ba3b3c20 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0ba3a5b40 con 0x55f0ba3b3c20 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb06ffd700 1 -- 172.21.15.66:0/4127512978 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb200147e0 con 0x55f0ba3b3c20 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb06ffd700 1 -- 172.21.15.66:0/4127512978 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb200149a0 con 0x55f0ba3b3c20 2023-12-11T19:34:54.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb4159eb80 10 client.25643 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0ba12fde0 con 0x55f0ba3b3c20 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb06ffd700 1 -- 172.21.15.66:0/4127512978 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f0ba3a5b40 con 0x55f0ba3b3c20 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb06ffd700 1 -- 172.21.15.66:0/4127512978 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb20023070 con 0x55f0ba3b3c20 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.920+0000 7ffb4159eb80 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb4159eb80 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb4159eb80 20 client.25643 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba3c7eb0 0x55f0ba3ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb4159eb80 4 client.25643 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:34:54.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0ba381f40 con 0x55f0ba3c7eb0 2023-12-11T19:34:54.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb277fe700 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba3c7eb0 0x55f0ba3ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb277fe700 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba3c7eb0 0x55f0ba3ca2d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc72460d32e49b04 server_cookie=e1b199cb9f0d50f7 in_seq=0 out_seq=0 2023-12-11T19:34:54.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.921+0000 7ffb06ffd700 10 client.25643 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb06ffd700 1 -- 172.21.15.66:0/4127512978 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0ba381f40 con 0x55f0ba3c7eb0 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb06ffd700 10 client.25643 handle_command_reply: tid=0 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb4159eb80 1 client.25643 shutdown 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 msgr2=0x55f0ba3c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c3760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000f570 tx=0x7ffb20003f50 comp rx=0 tx=0).stop 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 shutdown_connections 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba3c7eb0 0x55f0ba3ca2d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.922+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c3760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/4127512978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb06ffd700 0 client.25643 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 >> 172.21.15.66:0/4127512978 conn(0x55f0ba2e17e0 msgr2=0x55f0ba3ada40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 shutdown_connections 2023-12-11T19:34:54.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb4159eb80 1 -- 172.21.15.66:0/4127512978 wait complete. 2023-12-11T19:34:54.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.923+0000 7ffb4159eb80 20 client.25643 trim_cache size 0 max 16384 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 Processor -- start 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 -- start start 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba3a9b80 con 0x55f0ba3b2d30 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba3b1f70 con 0x55f0ba3b3c20 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.941+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba3bce60 con 0x55f0ba3b23d0 2023-12-11T19:34:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45016/0 (socket says 172.21.15.66:45016) 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 -- 172.21.15.66:0/3352283219 learned_addr learned my addr 172.21.15.66:0/3352283219 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb277fe700 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb27fff700 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 -- 172.21.15.66:0/3352283219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b23d0 msgr2=0x55f0ba3b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 -- 172.21.15.66:0/3352283219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 msgr2=0x55f0ba3bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.942+0000 7ffb26ffd700 1 -- 172.21.15.66:0/3352283219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0ba118c60 con 0x55f0ba3b2d30 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.943+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c3720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0041f0 tx=0x7ffb1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1243a461e724a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.943+0000 7ffb0e7fc700 1 -- 172.21.15.66:0/3352283219 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c003b10 con 0x55f0ba3b2d30 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.943+0000 7ffb0e7fc700 1 -- 172.21.15.66:0/3352283219 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb1c003c70 con 0x55f0ba3b2d30 2023-12-11T19:34:54.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.943+0000 7ffb0e7fc700 1 -- 172.21.15.66:0/3352283219 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0126b0 con 0x55f0ba3b2d30 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 -- 172.21.15.66:0/3352283219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 msgr2=0x55f0ba3c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c3720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0041f0 tx=0x7ffb1c014000 comp rx=0 tx=0).stop 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 -- 172.21.15.66:0/3352283219 shutdown_connections 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/3352283219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c3720 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 -- 172.21.15.66:0/3352283219 >> 172.21.15.66:0/3352283219 conn(0x55f0ba2e17e0 msgr2=0x55f0ba3ba1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 -- 172.21.15.66:0/3352283219 shutdown_connections 2023-12-11T19:34:54.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 -- 172.21.15.66:0/3352283219 wait complete. 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.944+0000 7ffb4159eb80 1 -- start start 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 Processor -- start 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba3c26d0 con 0x55f0ba3b23d0 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba3c2810 con 0x55f0ba3b3c20 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0ba3c29d0 con 0x55f0ba3b2d30 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb4159eb80 10 client.0 _ll_register_callbacks cb 0x55f0ba1eba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb27fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45024/0 (socket says 172.21.15.66:45024) 2023-12-11T19:34:54.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.945+0000 7ffb277fe700 1 -- 172.21.15.66:0/1008972838 learned_addr learned my addr 172.21.15.66:0/1008972838 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb277fe700 1 -- 172.21.15.66:0/1008972838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 msgr2=0x55f0ba3c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb277fe700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb277fe700 1 -- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 msgr2=0x55f0ba3c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb277fe700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb277fe700 1 -- 172.21.15.66:0/1008972838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c011e90 con 0x55f0ba3b23d0 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb27fff700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb277fe700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3c35b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffb180973b0 tx=0x7ffb18009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a216403c3d4ff8cb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:34:54.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb06ffd700 1 -- 172.21.15.66:0/1008972838 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180cb710 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb06ffd700 1 -- 172.21.15.66:0/1008972838 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb18099250 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb20004d10 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.946+0000 7ffb06ffd700 1 -- 172.21.15.66:0/1008972838 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb1809a3a0 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 10 client.15912 fetch_fsmap learned FSMap version 27 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0ba156250 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb06ffd700 1 -- 172.21.15.66:0/1008972838 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb20004d10 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb06ffd700 1 -- 172.21.15.66:0/1008972838 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb180cbac0 con 0x55f0ba3b23d0 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 20 client.15912 populate_metadata read hostname 'smithi066' 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba41f840 0x55f0ba421c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 4 client.15912 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:34:54.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.947+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f0ba2ccfd0 con 0x55f0ba41f840 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.948+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba41f840 0x55f0ba421c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.948+0000 7ffb26ffd700 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba41f840 0x55f0ba421c60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7073024ddb6e5023 server_cookie=be638662768c1074 in_seq=0 out_seq=0 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.948+0000 7ffb06ffd700 10 client.15912 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.948+0000 7ffb06ffd700 1 -- 172.21.15.66:0/1008972838 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f0ba2ccfd0 con 0x55f0ba41f840 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.948+0000 7ffb06ffd700 10 client.15912 handle_command_reply: tid=0 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.948+0000 7ffb4159eb80 1 client.15912 shutdown 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 msgr2=0x55f0ba3c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3c35b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffb180973b0 tx=0x7ffb18009380 comp rx=0 tx=0).stop 2023-12-11T19:34:54.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 shutdown_connections 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f0ba41f840 0x55f0ba421c60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f0ba3b3c20 0x55f0ba3c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f0ba3b23d0 0x55f0ba3c35b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 --2- 172.21.15.66:0/1008972838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0ba3b2d30 0x55f0ba3c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 >> 172.21.15.66:0/1008972838 conn(0x55f0ba2e17e0 msgr2=0x55f0ba3bc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb06ffd700 0 client.15912 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 shutdown_connections 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 1 -- 172.21.15.66:0/1008972838 wait complete. 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.949+0000 7ffb4159eb80 20 client.15912 trim_cache size 0 max 16384 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:34:54.955 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: "bb753821-e915-494e-a5bb-e5ddd58168b3": { 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "bb753821-e915-494e-a5bb-e5ddd58168b3", 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:34:54.956 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ffb100a1480 msgr2=0x7ffb100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ffb100a1480 0x7ffb100a3940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009cb0 tx=0x7ffb1c009260 comp rx=0 tx=0).stop 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 msgr2=0x7ffb28075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb28075330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009fd0 tx=0x7ffb1800fa00 comp rx=0 tx=0).stop 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 shutdown_connections 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ffb100a1480 0x7ffb100a3940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb28103130 0x7ffb28073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb281015a0 0x7ffb28075330 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 --2- 172.21.15.66:0/3109541071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28102790 0x7ffb280739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 >> 172.21.15.66:0/3109541071 conn(0x7ffb280fcb10 msgr2=0x7ffb28105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:34:54.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 shutdown_connections 2023-12-11T19:34:54.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:34:54.950+0000 7ffb2dbff700 1 -- 172.21.15.66:0/3109541071 wait complete. 2023-12-11T19:34:54.969 INFO:tasks.cephfs.filesystem:scrub status for tag:bb753821-e915-494e-a5bb-e5ddd58168b3 - {'path': '/', 'tag': 'bb753821-e915-494e-a5bb-e5ddd58168b3', 'options': 'recursive,force'} 2023-12-11T19:34:55.774 INFO:tasks.workunit.client.0.smithi066.stderr:Updating files: 62% (7550/12157) Updating files: 63% (7659/12157) Updating files: 64% (7781/12157) Updating files: 65% (7903/12157) Updating files: 66% (8024/12157) Updating files: 67% (8146/12157) Updating files: 68% (8267/12157) Updating files: 69% (8389/12157) Updating files: 70% (8510/12157) Updating files: 71% (8632/12157) Updating files: 72% (8754/12157) Updating files: 73% (8875/12157) Updating files: 74% (8997/12157) Updating files: 75% (9118/12157) Updating files: 76% (9240/12157) Updating files: 77% (9361/12157) Updating files: 78% (9483/12157) Updating files: 79% (9605/12157) Updating files: 80% (9726/12157) Updating files: 81% (9848/12157) Updating files: 82% (9969/12157) Updating files: 83% (10091/12157) Updating files: 84% (10212/12157) Updating files: 85% (10334/12157) Updating files: 86% (10456/12157) Updating files: 87% (10577/12157) Updating files: 88% (10699/12157) Updating files: 89% (10820/12157) Updating files: 90% (10942/12157) Updating files: 91% (11063/12157) Updating files: 92% (11185/12157) Updating files: 93% (11307/12157) Updating files: 94% (11428/12157) Updating files: 95% (11550/12157) Updating files: 96% (11671/12157) Updating files: 97% (11793/12157) Updating files: 98% (11914/12157) Updating files: 99% (12036/12157) Updating files: 100% (12157/12157) Updating files: 100% (12157/12157), done. 2023-12-11T19:34:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:55 smithi167 ceph-mon[102750]: pgmap v400: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2023-12-11T19:34:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:55 smithi167 ceph-mon[102750]: scrub queued for path: / 2023-12-11T19:34:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:55 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:55 smithi167 ceph-mon[102750]: scrub summary: active paths [/] 2023-12-11T19:34:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:55 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:56.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:55 smithi066 ceph-mon[97920]: pgmap v400: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2023-12-11T19:34:56.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:55 smithi066 ceph-mon[97920]: scrub queued for path: / 2023-12-11T19:34:56.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:55 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:56.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:55 smithi066 ceph-mon[97920]: scrub summary: active paths [/] 2023-12-11T19:34:56.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:55 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:55 smithi070 ceph-mon[104044]: pgmap v400: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2023-12-11T19:34:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:55 smithi070 ceph-mon[104044]: scrub queued for path: / 2023-12-11T19:34:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:55 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:55 smithi070 ceph-mon[104044]: scrub summary: active paths [/] 2023-12-11T19:34:56.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:55 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr:Note: switching to 'a62fda73efba5520d409bb5a21bd8373289dda5d'. 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr:state without impacting any branches by switching back to a branch. 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-11T19:34:56.356 INFO:tasks.workunit.client.0.smithi066.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: git switch -c 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr:Or undo this operation with: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: git switch - 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-12-11T19:34:56.357 INFO:tasks.workunit.client.0.smithi066.stderr:HEAD is now at a62fda73efb Merge remote-tracking branch 'milind/mds-enqueue-all-child-frags-for-scrub-for-a-given-fragset' into rishabh-2023dec11 2023-12-11T19:34:56.363 DEBUG:teuthology.orchestra.run.smithi066:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-11T19:34:56.456 INFO:tasks.workunit.client.0.smithi066.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-11T19:34:56.461 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-11T19:34:56.461 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-11T19:34:57.008 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-11T19:34:57.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:56 smithi167 ceph-mon[102750]: scrub summary: idle 2023-12-11T19:34:57.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:56 smithi066 ceph-mon[97920]: scrub summary: idle 2023-12-11T19:34:57.101 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-11T19:34:57.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:56 smithi070 ceph-mon[104044]: scrub summary: idle 2023-12-11T19:34:57.168 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-11T19:34:57.172 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-11T19:34:57.173 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-11T19:34:57.239 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-11T19:34:57.246 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:34:57.246 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-11T19:34:57.291 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2023-12-11T19:34:57.292 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2023-12-11T19:34:57.292 DEBUG:teuthology.orchestra.run.smithi066:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a62fda73efba5520d409bb5a21bd8373289dda5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2023-12-11T19:34:57.398 INFO:tasks.workunit.client.0.smithi066.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2023-12-11T19:34:57.455 INFO:tasks.workunit.client.0.smithi066.stdout: Iozone: Performance Test of File I/O 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Version $Revision: 3.489 $ 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Compiled for 64 bit mode. 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Build: linux-AMD64 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: Alexey Skidanov, Sudhir Kumar. 2023-12-11T19:34:57.456 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Run began: Mon Dec 11 19:34:57 2023 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Include close in write timing 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Include fsync in write timing 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: File size set to 1048576 kB 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Record Size 16 kB 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Output is in kBytes/sec 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Time Resolution = 0.000001 seconds. 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Processor cache size set to 1024 kBytes. 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Processor cache line size set to 32 bytes. 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: File stride size set to 17 * record size. 2023-12-11T19:34:57.457 INFO:tasks.workunit.client.0.smithi066.stdout: Throughput test with 1 process 2023-12-11T19:34:57.458 INFO:tasks.workunit.client.0.smithi066.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2023-12-11T19:34:58.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:57 smithi167 ceph-mon[102750]: pgmap v401: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:58.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:57 smithi066 ceph-mon[97920]: pgmap v401: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:34:58.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:57 smithi070 ceph-mon[104044]: pgmap v401: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-11T19:35:00.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:34:59 smithi167 ceph-mon[102750]: pgmap v402: 97 pgs: 97 active+clean; 13 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1024 KiB/s wr, 10 op/s 2023-12-11T19:35:00.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:34:59 smithi066 ceph-mon[97920]: pgmap v402: 97 pgs: 97 active+clean; 13 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1024 KiB/s wr, 10 op/s 2023-12-11T19:35:00.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:34:59 smithi070 ceph-mon[104044]: pgmap v402: 97 pgs: 97 active+clean; 13 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1024 KiB/s wr, 10 op/s 2023-12-11T19:35:02.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:01 smithi167 ceph-mon[102750]: pgmap v403: 97 pgs: 97 active+clean; 73 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 MiB/s wr, 15 op/s 2023-12-11T19:35:02.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:01 smithi066 ceph-mon[97920]: pgmap v403: 97 pgs: 97 active+clean; 73 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 MiB/s wr, 15 op/s 2023-12-11T19:35:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:01 smithi070 ceph-mon[104044]: pgmap v403: 97 pgs: 97 active+clean; 73 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 MiB/s wr, 15 op/s 2023-12-11T19:35:04.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:03 smithi167 ceph-mon[102750]: pgmap v404: 97 pgs: 97 active+clean; 105 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 MiB/s wr, 13 op/s 2023-12-11T19:35:04.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:03 smithi066 ceph-mon[97920]: pgmap v404: 97 pgs: 97 active+clean; 105 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 MiB/s wr, 13 op/s 2023-12-11T19:35:04.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:03 smithi070 ceph-mon[104044]: pgmap v404: 97 pgs: 97 active+clean; 105 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.7 MiB/s wr, 13 op/s 2023-12-11T19:35:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 -- 172.21.15.66:0/3544176427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0681034a0 msgr2=0x7fa0681038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 --2- 172.21.15.66:0/3544176427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0681034a0 0x7fa0681038c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b0d0 tx=0x7fa06400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 -- 172.21.15.66:0/3544176427 shutdown_connections 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 --2- 172.21.15.66:0/3544176427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa068105030 0x7fa06810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 --2- 172.21.15.66:0/3544176427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0681034a0 0x7fa0681038c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 --2- 172.21.15.66:0/3544176427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa068104690 0x7fa068104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 -- 172.21.15.66:0/3544176427 >> 172.21.15.66:0/3544176427 conn(0x7fa0680fea30 msgr2=0x7fa068100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 -- 172.21.15.66:0/3544176427 shutdown_connections 2023-12-11T19:35:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.426+0000 7fa06f39d700 1 -- 172.21.15.66:0/3544176427 wait complete. 2023-12-11T19:35:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 Processor -- start 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 -- start start 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0681034a0 0x7fa068199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa068104690 0x7fa0681999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 0x7fa0681a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681a0f70 con 0x7fa068105030 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681a10b0 con 0x7fa068104690 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.428+0000 7fa06f39d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681a1240 con 0x7fa0681034a0 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 0x7fa0681a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0681034a0 0x7fa068199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 0x7fa0681a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39200/0 (socket says 172.21.15.66:39200) 2023-12-11T19:35:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 -- 172.21.15.66:0/36411388 learned_addr learned my addr 172.21.15.66:0/36411388 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0681034a0 0x7fa068199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:45286/0 (socket says 172.21.15.66:45286) 2023-12-11T19:35:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06db9a700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa068104690 0x7fa0681999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 -- 172.21.15.66:0/36411388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0681034a0 msgr2=0x7fa068199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0681034a0 0x7fa068199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 -- 172.21.15.66:0/36411388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa068104690 msgr2=0x7fa0681999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa068104690 0x7fa0681999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.429+0000 7fa06eb9c700 1 -- 172.21.15.66:0/36411388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009d00 con 0x7fa068105030 2023-12-11T19:35:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.430+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 0x7fa0681a09a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d980 tx=0x7fa06000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.430+0000 7fa05f7fe700 1 -- 172.21.15.66:0/36411388 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060004a30 con 0x7fa068105030 2023-12-11T19:35:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.430+0000 7fa05f7fe700 1 -- 172.21.15.66:0/36411388 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0600054b0 con 0x7fa068105030 2023-12-11T19:35:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.430+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0681a1520 con 0x7fa068105030 2023-12-11T19:35:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.430+0000 7fa05f7fe700 1 -- 172.21.15.66:0/36411388 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000fab0 con 0x7fa068105030 2023-12-11T19:35:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.430+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0681a19e0 con 0x7fa068105030 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.433+0000 7fa05f7fe700 1 -- 172.21.15.66:0/36411388 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa06000fc10 con 0x7fa068105030 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.435+0000 7fa05f7fe700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa0540a1470 0x7fa0540a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.435+0000 7fa05f7fe700 1 -- 172.21.15.66:0/36411388 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa0600ccb10 con 0x7fa068105030 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.435+0000 7fa06e39b700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa0540a1470 0x7fa0540a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.436+0000 7fa083f43b80 1 Processor -- start 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.436+0000 7fa06e39b700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa0540a1470 0x7fa0540a3930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa064007a30 tx=0x7fa06400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.436+0000 7fa083f43b80 1 -- start start 2023-12-11T19:35:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e30931ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e30931f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931fb80 0x55e309321f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa083f43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e309275af0 con 0x55e30931b0d0 2023-12-11T19:35:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa083f43b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e309273410 con 0x55e30931a770 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa083f43b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30926ed00 con 0x55e30931fb80 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa06db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e30931f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa06db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e30931f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39210/0 (socket says 172.21.15.66:39210) 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa06db9a700 1 -- 172.21.15.66:0/367573813 learned_addr learned my addr 172.21.15.66:0/367573813 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931fb80 0x55e309321f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.437+0000 7fa06e39b700 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e30931ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06db9a700 1 -- 172.21.15.66:0/367573813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931fb80 msgr2=0x55e309321f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06db9a700 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931fb80 0x55e309321f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06db9a700 1 -- 172.21.15.66:0/367573813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 msgr2=0x55e30931ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06db9a700 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e30931ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06db9a700 1 -- 172.21.15.66:0/367573813 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3090eb5d0 con 0x55e30931b0d0 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931fb80 0x55e309321f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa06db9a700 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e30931f640 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa058007d70 tx=0x7fa0580106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890f3f6e6be03993 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa05cbfe700 1 -- 172.21.15.66:0/367573813 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058010e10 con 0x55e30931b0d0 2023-12-11T19:35:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa05cbfe700 1 -- 172.21.15.66:0/367573813 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa058015070 con 0x55e30931b0d0 2023-12-11T19:35:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.438+0000 7fa05cbfe700 1 -- 172.21.15.66:0/367573813 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058003dc0 con 0x55e30931b0d0 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 -- 172.21.15.66:0/367573813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 msgr2=0x55e30931f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e30931f640 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa058007d70 tx=0x7fa0580106b0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 -- 172.21.15.66:0/367573813 shutdown_connections 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e30931ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931fb80 0x55e309321f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 --2- 172.21.15.66:0/367573813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e30931f640 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.439+0000 7fa083f43b80 1 -- 172.21.15.66:0/367573813 >> 172.21.15.66:0/367573813 conn(0x55e309249780 msgr2=0x55e309316b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.440+0000 7fa083f43b80 1 -- 172.21.15.66:0/367573813 shutdown_connections 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.440+0000 7fa083f43b80 1 -- 172.21.15.66:0/367573813 wait complete. 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.440+0000 7fa083f43b80 1 -- start start 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.440+0000 7fa083f43b80 1 Processor -- start 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 0x55e309323f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e309324440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e309228500 con 0x55e30931a770 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3092281a0 con 0x55e30931fb80 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3090ed8c0 con 0x55e30931b0d0 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa083f43b80 10 client.0 _ll_register_callbacks cb 0x55e309153a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa06e39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 0x55e309323f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa06e39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 0x55e309323f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39222/0 (socket says 172.21.15.66:39222) 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.441+0000 7fa06e39b700 1 -- 172.21.15.66:0/500859996 learned_addr learned my addr 172.21.15.66:0/500859996 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06db9a700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e309324440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06e39b700 1 -- 172.21.15.66:0/500859996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 msgr2=0x55e309324440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06e39b700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e309324440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06e39b700 1 -- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 msgr2=0x55e30932b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06e39b700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06e39b700 1 -- 172.21.15.66:0/500859996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0681a19e0 con 0x55e30931a770 2023-12-11T19:35:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa06e39b700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 0x55e309323f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b0d0 tx=0x7fa0640099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0a1b0f10c97c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.442+0000 7fa051ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa051ffb700 1 -- 172.21.15.66:0/500859996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064003e20 con 0x55e30931a770 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa051ffb700 1 -- 172.21.15.66:0/500859996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa064022040 con 0x55e30931a770 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e30930d730 con 0x55e30931a770 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa051ffb700 1 -- 172.21.15.66:0/500859996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa064005940 con 0x55e30931a770 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa083f43b80 10 client.15924 fetch_fsmap learned FSMap version 27 2023-12-11T19:35:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa0580079a0 con 0x55e30931a770 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.443+0000 7fa051ffb700 1 -- 172.21.15.66:0/500859996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e30930d730 con 0x55e30931a770 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa051ffb700 1 -- 172.21.15.66:0/500859996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa06401c440 con 0x55e30931a770 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa083f43b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa083f43b80 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa083f43b80 20 client.15924 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa083f43b80 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e30932fd40 0x55e309332160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa083f43b80 4 client.15924 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3092e9ee0 con 0x55e30932fd40 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.444+0000 7fa06db9a700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e30932fd40 0x55e309332160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.445+0000 7fa06db9a700 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e30932fd40 0x55e309332160 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306173b65a628b2e server_cookie=93cb230eabaaf0d5 in_seq=0 out_seq=0 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.445+0000 7fa051ffb700 10 client.15924 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.445+0000 7fa051ffb700 1 -- 172.21.15.66:0/500859996 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3092e9ee0 con 0x55e30932fd40 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.445+0000 7fa051ffb700 10 client.15924 handle_command_reply: tid=0 2023-12-11T19:35:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.445+0000 7fa083f43b80 1 client.15924 shutdown 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.445+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 msgr2=0x55e309323f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 0x55e309323f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b0d0 tx=0x7fa0640099e0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 shutdown_connections 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e30932fd40 0x55e309332160 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931a770 0x55e309323f00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa051ffb700 0 client.15924 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 --2- 172.21.15.66:0/500859996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e309324440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 >> 172.21.15.66:0/500859996 conn(0x55e309249780 msgr2=0x55e309317560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 shutdown_connections 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 1 -- 172.21.15.66:0/500859996 wait complete. 2023-12-11T19:35:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.446+0000 7fa083f43b80 20 client.15924 trim_cache size 0 max 16384 2023-12-11T19:35:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 Processor -- start 2023-12-11T19:35:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 -- start start 2023-12-11T19:35:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e309332130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e30932bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 0x55e309323410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e309315590 con 0x55e30931fb80 2023-12-11T19:35:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30931f000 con 0x55e30931a770 2023-12-11T19:35:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.463+0000 7fa083f43b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30931c8f0 con 0x55e30931b0d0 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 0x55e309323410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 0x55e309323410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39224/0 (socket says 172.21.15.66:39224) 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2353920902 learned_addr learned my addr 172.21.15.66:0/2353920902 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06e39b700 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e309332130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06db9a700 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e30932bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2353920902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 msgr2=0x55e30932bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e30932bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2353920902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 msgr2=0x55e309332130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e309332130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2353920902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e309097d80 con 0x55e30931fb80 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.464+0000 7fa06e39b700 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e309332130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.465+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 0x55e309323410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa0600962f0 tx=0x7fa060005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518fbc0820c5cc92 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.465+0000 7fa05d3ff700 1 -- 172.21.15.66:0/2353920902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060099df0 con 0x55e30931fb80 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.465+0000 7fa05d3ff700 1 -- 172.21.15.66:0/2353920902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa06009df40 con 0x55e30931fb80 2023-12-11T19:35:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.465+0000 7fa05d3ff700 1 -- 172.21.15.66:0/2353920902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0600adbd0 con 0x55e30931fb80 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 -- 172.21.15.66:0/2353920902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 msgr2=0x55e309323410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 0x55e309323410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa0600962f0 tx=0x7fa060005620 comp rx=0 tx=0).stop 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 -- 172.21.15.66:0/2353920902 shutdown_connections 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931a770 0x55e309332130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931fb80 0x55e309323410 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2353920902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931b0d0 0x55e30932bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 -- 172.21.15.66:0/2353920902 >> 172.21.15.66:0/2353920902 conn(0x55e309249780 msgr2=0x55e30931f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 -- 172.21.15.66:0/2353920902 shutdown_connections 2023-12-11T19:35:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 -- 172.21.15.66:0/2353920902 wait complete. 2023-12-11T19:35:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.466+0000 7fa083f43b80 1 -- start start 2023-12-11T19:35:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.467+0000 7fa083f43b80 1 Processor -- start 2023-12-11T19:35:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931a770 0x55e309328c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e309329170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30932a6c0 con 0x55e30931b0d0 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30932a830 con 0x55e30931fb80 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30932a9f0 con 0x55e30931a770 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa083f43b80 10 client.0 _ll_register_callbacks cb 0x55e309153a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa06db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e309329170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa06eb9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa06e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931a770 0x55e309328c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa06e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931a770 0x55e309328c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:45326/0 (socket says 172.21.15.66:45326) 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa06eb9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:52648/0 (socket says 172.21.15.66:52648) 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.468+0000 7fa06e39b700 1 -- 172.21.15.66:0/2145032788 learned_addr learned my addr 172.21.15.66:0/2145032788 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2145032788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931a770 msgr2=0x55e309328c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931a770 0x55e309328c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2145032788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 msgr2=0x55e309329170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e309329170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06eb9c700 1 -- 172.21.15.66:0/2145032788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0580079a0 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06db9a700 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e309329170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa06eb9c700 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932a180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55e309337520 tx=0x7fa0600997c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fcfdab72f0298d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa051ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa051ffb700 1 -- 172.21.15.66:0/2145032788 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060099df0 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa051ffb700 1 -- 172.21.15.66:0/2145032788 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa06009d040 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.469+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa058015070 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa051ffb700 1 -- 172.21.15.66:0/2145032788 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0600a2180 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 10 client.25667 fetch_fsmap learned FSMap version 27 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa0681a1520 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa051ffb700 1 -- 172.21.15.66:0/2145032788 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa058015070 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa051ffb700 1 -- 172.21.15.66:0/2145032788 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa060095be0 con 0x55e30931fb80 2023-12-11T19:35:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 27 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 20 client.25667 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e309386790 0x55e309388bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 4 client.25667 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.470+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e309234f70 con 0x55e309386790 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.471+0000 7fa06e39b700 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e309386790 0x55e309388bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.472+0000 7fa06e39b700 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e309386790 0x55e309388bb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6022489b553f5c7 server_cookie=c44b4089bfca5e15 in_seq=0 out_seq=0 2023-12-11T19:35:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.472+0000 7fa051ffb700 10 client.25667 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa051ffb700 1 -- 172.21.15.66:0/2145032788 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e309234f70 con 0x55e309386790 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa051ffb700 10 client.25667 handle_command_reply: tid=0 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 client.25667 shutdown 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 msgr2=0x55e30932a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932a180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55e309337520 tx=0x7fa0600997c0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 shutdown_connections 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55e309386790 0x55e309388bb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e30931fb80 0x55e30932a180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa051ffb700 0 client.25667 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e30931a770 0x55e309328c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 --2- 172.21.15.66:0/2145032788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e30931b0d0 0x55e309329170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 >> 172.21.15.66:0/2145032788 conn(0x55e309249780 msgr2=0x55e30931f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 shutdown_connections 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 1 -- 172.21.15.66:0/2145032788 wait complete. 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.482+0000 7fa083f43b80 20 client.25667 trim_cache size 0 max 16384 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:35:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:35:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:35:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa0540a1470 msgr2=0x7fa0540a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa0540a1470 0x7fa0540a3930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa064007a30 tx=0x7fa06400b040 comp rx=0 tx=0).stop 2023-12-11T19:35:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 msgr2=0x7fa0681a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 0x7fa0681a09a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d980 tx=0x7fa06000dc90 comp rx=0 tx=0).stop 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 shutdown_connections 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fa0540a1470 0x7fa0540a3930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa068105030 0x7fa0681a09a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0681034a0 0x7fa068199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 --2- 172.21.15.66:0/36411388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa068104690 0x7fa0681999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 >> 172.21.15.66:0/36411388 conn(0x7fa0680fea30 msgr2=0x7fa068107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 shutdown_connections 2023-12-11T19:35:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:05.483+0000 7fa06f39d700 1 -- 172.21.15.66:0/36411388 wait complete. 2023-12-11T19:35:06.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:05 smithi167 ceph-mon[102750]: pgmap v405: 97 pgs: 97 active+clean; 225 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 19 MiB/s wr, 17 op/s 2023-12-11T19:35:06.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:05 smithi066 ceph-mon[97920]: pgmap v405: 97 pgs: 97 active+clean; 225 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 19 MiB/s wr, 17 op/s 2023-12-11T19:35:06.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:05 smithi070 ceph-mon[104044]: pgmap v405: 97 pgs: 97 active+clean; 225 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 19 MiB/s wr, 17 op/s 2023-12-11T19:35:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:07 smithi066 ceph-mon[97920]: pgmap v406: 97 pgs: 97 active+clean; 333 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2023-12-11T19:35:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:35:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:07 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:07 smithi070 ceph-mon[104044]: pgmap v406: 97 pgs: 97 active+clean; 333 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2023-12-11T19:35:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:35:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:07.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:07 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:08.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:07 smithi167 ceph-mon[102750]: pgmap v406: 97 pgs: 97 active+clean; 333 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2023-12-11T19:35:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:35:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:07 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:08.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:08 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:35:08.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:08 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:35:08.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:08 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:08.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:08 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:35:08.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:08 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:35:08.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:08 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:08 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:35:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:08 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:35:09.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:08 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:35:09.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:09 smithi066 ceph-mon[97920]: pgmap v407: 97 pgs: 97 active+clean; 425 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 35 MiB/s wr, 21 op/s 2023-12-11T19:35:09.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:09 smithi070 ceph-mon[104044]: pgmap v407: 97 pgs: 97 active+clean; 425 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 35 MiB/s wr, 21 op/s 2023-12-11T19:35:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:09 smithi167 ceph-mon[102750]: pgmap v407: 97 pgs: 97 active+clean; 425 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 35 MiB/s wr, 21 op/s 2023-12-11T19:35:11.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:11 smithi066 ceph-mon[97920]: pgmap v408: 97 pgs: 97 active+clean; 557 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-12-11T19:35:11.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:11 smithi070 ceph-mon[104044]: pgmap v408: 97 pgs: 97 active+clean; 557 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-12-11T19:35:12.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:11 smithi167 ceph-mon[102750]: pgmap v408: 97 pgs: 97 active+clean; 557 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-12-11T19:35:12.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:12 smithi066 ceph-mon[97920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:12.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:12 smithi070 ceph-mon[104044]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:13.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:12 smithi167 ceph-mon[102750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:13.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:13 smithi066 ceph-mon[97920]: pgmap v409: 97 pgs: 97 active+clean; 633 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-12-11T19:35:13.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:13 smithi070 ceph-mon[104044]: pgmap v409: 97 pgs: 97 active+clean; 633 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-12-11T19:35:14.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:13 smithi167 ceph-mon[102750]: pgmap v409: 97 pgs: 97 active+clean; 633 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-12-11T19:35:15.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:15 smithi066 ceph-mon[97920]: pgmap v410: 97 pgs: 97 active+clean; 805 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 58 MiB/s wr, 28 op/s 2023-12-11T19:35:15.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:15 smithi070 ceph-mon[104044]: pgmap v410: 97 pgs: 97 active+clean; 805 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 58 MiB/s wr, 28 op/s 2023-12-11T19:35:16.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:15 smithi167 ceph-mon[102750]: pgmap v410: 97 pgs: 97 active+clean; 805 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 58 MiB/s wr, 28 op/s 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 initial writers = 55971.06 kB/sec 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 initial writers = 55968.80 kB/sec 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 55971.06 kB/sec 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 55971.06 kB/sec 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 55971.06 kB/sec 2023-12-11T19:35:16.197 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:35:17.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:17 smithi066 ceph-mon[97920]: pgmap v411: 97 pgs: 97 active+clean; 901 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 56 MiB/s wr, 28 op/s 2023-12-11T19:35:17.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:17 smithi070 ceph-mon[104044]: pgmap v411: 97 pgs: 97 active+clean; 901 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 56 MiB/s wr, 28 op/s 2023-12-11T19:35:18.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:17 smithi167 ceph-mon[102750]: pgmap v411: 97 pgs: 97 active+clean; 901 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 56 MiB/s wr, 28 op/s 2023-12-11T19:35:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 -- 172.21.15.66:0/1333984108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 msgr2=0x7fd8f0101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/1333984108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f0101950 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00099c0 tx=0x7fd8e0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 -- 172.21.15.66:0/1333984108 shutdown_connections 2023-12-11T19:35:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/1333984108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f01030c0 0x7fd8f0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/1333984108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f0101950 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/1333984108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f0102720 0x7fd8f0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 -- 172.21.15.66:0/1333984108 >> 172.21.15.66:0/1333984108 conn(0x7fd8f00fcaa0 msgr2=0x7fd8f00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 -- 172.21.15.66:0/1333984108 shutdown_connections 2023-12-11T19:35:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.652+0000 7fd8f57ff700 1 -- 172.21.15.66:0/1333984108 wait complete. 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.653+0000 7fd8f57ff700 1 Processor -- start 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.653+0000 7fd8f57ff700 1 -- start start 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8f57ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8f57ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0102720 0x7fd8f0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8f57ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f01030c0 0x7fd8f01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8f57ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01a1180 con 0x7fd8f0101530 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8f57ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01a12c0 con 0x7fd8f0102720 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8f57ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01a1450 con 0x7fd8f01030c0 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33630/0 (socket says 172.21.15.66:33630) 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 -- 172.21.15.66:0/4236531440 learned_addr learned my addr 172.21.15.66:0/4236531440 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0102720 0x7fd8f0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f01030c0 0x7fd8f01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 -- 172.21.15.66:0/4236531440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f01030c0 msgr2=0x7fd8f01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f01030c0 0x7fd8f01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 -- 172.21.15.66:0/4236531440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0102720 msgr2=0x7fd8f0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0102720 0x7fd8f0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8eeffd700 1 -- 172.21.15.66:0/4236531440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00095f0 con 0x7fd8f0101530 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.654+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f01030c0 0x7fd8f01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.655+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f01996a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009fd0 tx=0x7fd8e000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.655+0000 7fd8d7fff700 1 -- 172.21.15.66:0/4236531440 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e001d070 con 0x7fd8f0101530 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.655+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f01a16d0 con 0x7fd8f0101530 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.655+0000 7fd8d7fff700 1 -- 172.21.15.66:0/4236531440 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8e000bd10 con 0x7fd8f0101530 2023-12-11T19:35:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.655+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f01a1b00 con 0x7fd8f0101530 2023-12-11T19:35:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.655+0000 7fd8d7fff700 1 -- 172.21.15.66:0/4236531440 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0004b40 con 0x7fd8f0101530 2023-12-11T19:35:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.658+0000 7fd8d7fff700 1 -- 172.21.15.66:0/4236531440 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd8e0004ca0 con 0x7fd8f0101530 2023-12-11T19:35:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.659+0000 7fd9090ffb80 1 Processor -- start 2023-12-11T19:35:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 -- start start 2023-12-11T19:35:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd8d7fff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd8d80a1480 0x7fd8d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd8d7fff700 1 -- 172.21.15.66:0/4236531440 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8e00cda30 con 0x7fd8f0101530 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d7a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd8d80a1480 0x7fd8d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d7aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8cd4af0 con 0x55aec8d79830 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8cd2410 con 0x55aec8d7a720 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8ccdd00 con 0x55aec8d78ed0 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d7a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.660+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d7a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33642/0 (socket says 172.21.15.66:33642) 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:59666/0 (socket says 172.21.15.66:59666) 2023-12-11T19:35:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8ee7fc700 1 -- 172.21.15.66:0/1985114254 learned_addr learned my addr 172.21.15.66:0/1985114254 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d7aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 -- 172.21.15.66:0/1985114254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 msgr2=0x55aec8d7aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d7aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 -- 172.21.15.66:0/1985114254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 msgr2=0x55aec8d7a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d7a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 -- 172.21.15.66:0/1985114254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aec8b4a5d0 con 0x55aec8d78ed0 2023-12-11T19:35:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd8d80a1480 0x7fd8d80a3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80060b0 tx=0x7fd8e8007a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d7a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.661+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d792f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8e009aaa0 tx=0x7fd8e009a9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7b3933b5e481d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd8d57fa700 1 -- 172.21.15.66:0/1985114254 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e009e070 con 0x55aec8d78ed0 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd8d57fa700 1 -- 172.21.15.66:0/1985114254 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8e009b4a0 con 0x55aec8d78ed0 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd8d57fa700 1 -- 172.21.15.66:0/1985114254 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e009a290 con 0x55aec8d78ed0 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1985114254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 msgr2=0x55aec8d792f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d792f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8e009aaa0 tx=0x7fd8e009a9e0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1985114254 shutdown_connections 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d7aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d792f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1985114254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d7a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.662+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1985114254 >> 172.21.15.66:0/1985114254 conn(0x55aec8ca8780 msgr2=0x55aec8d75880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.663+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1985114254 shutdown_connections 2023-12-11T19:35:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.663+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1985114254 wait complete. 2023-12-11T19:35:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.663+0000 7fd9090ffb80 1 -- start start 2023-12-11T19:35:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.663+0000 7fd9090ffb80 1 Processor -- start 2023-12-11T19:35:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 0x55aec8d830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 0x55aec8d89ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8c87500 con 0x55aec8d7a720 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8c871a0 con 0x55aec8d79830 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8b4c8c0 con 0x55aec8d78ed0 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd9090ffb80 10 client.0 _ll_register_callbacks cb 0x55aec8bb2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8ef7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 0x55aec8d89ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 0x55aec8d830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 0x55aec8d830a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:40168/0 (socket says 172.21.15.66:40168) 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8ee7fc700 1 -- 172.21.15.66:0/1952381587 learned_addr learned my addr 172.21.15.66:0/1952381587 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d82b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:59678/0 (socket says 172.21.15.66:59678) 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.664+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 0x55aec8d89ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33658/0 (socket says 172.21.15.66:33658) 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ee7fc700 1 -- 172.21.15.66:0/1952381587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 msgr2=0x55aec8d82b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d82b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ee7fc700 1 -- 172.21.15.66:0/1952381587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 msgr2=0x55aec8d89ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 0x55aec8d89ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ee7fc700 1 -- 172.21.15.66:0/1952381587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e40079a0 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 0x55aec8d89ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 0x55aec8d830a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005fb0 tx=0x7fd8e40119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403997800930a75 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/1952381587 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4016070 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.665+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aec8d6c730 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/1952381587 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8e4004830 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/1952381587 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e40049f0 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 10 client.25679 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fd8f01a1b00 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/1952381587 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aec8d6c730 con 0x55aec8d79830 2023-12-11T19:35:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/1952381587 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd8e4010640 con 0x55aec8d79830 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 10 client.25679 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 20 client.25679 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8d8e9f0 0x55aec8d90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.666+0000 7fd9090ffb80 4 client.25679 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.667+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aec8d48ee0 con 0x55aec8d8e9f0 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.667+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8d8e9f0 0x55aec8d90e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.667+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8d8e9f0 0x55aec8d90e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c36478a2e4d4e99 server_cookie=5d5913db90eab7f2 in_seq=0 out_seq=0 2023-12-11T19:35:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.667+0000 7fd8d2ff5700 10 client.25679 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/1952381587 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aec8d48ee0 con 0x55aec8d8e9f0 2023-12-11T19:35:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd8d2ff5700 10 client.25679 handle_command_reply: tid=0 2023-12-11T19:35:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 client.25679 shutdown 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 msgr2=0x55aec8d830a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 0x55aec8d830a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4005fb0 tx=0x7fd8e40119a0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 shutdown_connections 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8d8e9f0 0x55aec8d90e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d7a720 0x55aec8d89ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.668+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.669+0000 7fd8d2ff5700 0 client.25679 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.669+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/1952381587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d79830 0x55aec8d830a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.669+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 >> 172.21.15.66:0/1952381587 conn(0x55aec8ca8780 msgr2=0x55aec8d73ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.669+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 shutdown_connections 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.669+0000 7fd9090ffb80 1 -- 172.21.15.66:0/1952381587 wait complete. 2023-12-11T19:35:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.669+0000 7fd9090ffb80 20 client.25679 trim_cache size 0 max 16384 2023-12-11T19:35:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 Processor -- start 2023-12-11T19:35:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 -- start start 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 0x55aec8d81e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d79830 0x55aec8d89df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d80c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8d824c0 con 0x55aec8d78ed0 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8d83850 con 0x55aec8d7a720 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8d74d70 con 0x55aec8d79830 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 0x55aec8d81e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd8eeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 0x55aec8d81e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33660/0 (socket says 172.21.15.66:33660) 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.685+0000 7fd8eeffd700 1 -- 172.21.15.66:0/3776791227 learned_addr learned my addr 172.21.15.66:0/3776791227 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d80c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d79830 0x55aec8d89df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8eeffd700 1 -- 172.21.15.66:0/3776791227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d79830 msgr2=0x55aec8d89df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d79830 0x55aec8d89df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8eeffd700 1 -- 172.21.15.66:0/3776791227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 msgr2=0x55aec8d80c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d80c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8eeffd700 1 -- 172.21.15.66:0/3776791227 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aec8af6d80 con 0x55aec8d78ed0 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d79830 0x55aec8d89df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 0x55aec8d81e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000c00 tx=0x7fd8e000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc8c916aef1aa60 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8d5ffb700 1 -- 172.21.15.66:0/3776791227 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e009e040 con 0x55aec8d78ed0 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8d5ffb700 1 -- 172.21.15.66:0/3776791227 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8e00a30a0 con 0x55aec8d78ed0 2023-12-11T19:35:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.686+0000 7fd8d5ffb700 1 -- 172.21.15.66:0/3776791227 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0099260 con 0x55aec8d78ed0 2023-12-11T19:35:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 -- 172.21.15.66:0/3776791227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 msgr2=0x55aec8d81e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 0x55aec8d81e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000c00 tx=0x7fd8e000b540 comp rx=0 tx=0).stop 2023-12-11T19:35:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 -- 172.21.15.66:0/3776791227 shutdown_connections 2023-12-11T19:35:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d80c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d78ed0 0x55aec8d81e60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/3776791227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d79830 0x55aec8d89df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 -- 172.21.15.66:0/3776791227 >> 172.21.15.66:0/3776791227 conn(0x55aec8ca8780 msgr2=0x55aec8d816b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 -- 172.21.15.66:0/3776791227 shutdown_connections 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 -- 172.21.15.66:0/3776791227 wait complete. 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.687+0000 7fd9090ffb80 1 -- start start 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 Processor -- start 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d87540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d87a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d88970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8d88eb0 con 0x55aec8d79830 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8d89020 con 0x55aec8d7a720 2023-12-11T19:35:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aec8d891e0 con 0x55aec8d78ed0 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd9090ffb80 10 client.0 _ll_register_callbacks cb 0x55aec8bb2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d87a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d87a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33668/0 (socket says 172.21.15.66:33668) 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd8ee7fc700 1 -- 172.21.15.66:0/874193226 learned_addr learned my addr 172.21.15.66:0/874193226 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.688+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d87540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8ef7fe700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d88970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8eeffd700 1 -- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 msgr2=0x55aec8d88970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d88970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8eeffd700 1 -- 172.21.15.66:0/874193226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 msgr2=0x55aec8d87a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d87a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8eeffd700 1 -- 172.21.15.66:0/874193226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aec8b4a5d0 con 0x55aec8d78ed0 2023-12-11T19:35:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d87a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8eeffd700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d87540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000c00 tx=0x7fd8e00b90d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a2e125104ed0c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/874193226 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e009e040 con 0x55aec8d78ed0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/874193226 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8e00a2e10 con 0x55aec8d78ed0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.689+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8e009b4a0 con 0x55aec8d78ed0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/874193226 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e0099240 con 0x55aec8d78ed0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 10 client.25675 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55aec8adfc00 con 0x55aec8d78ed0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/874193226 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8e009b4a0 con 0x55aec8d78ed0 2023-12-11T19:35:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/874193226 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd8e0099e00 con 0x55aec8d78ed0 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 10 client.25675 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 10 client.25675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 20 client.25675 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8de5140 0x55aec8de7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 4 client.25675 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.690+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aec8c93f70 con 0x55aec8de5140 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.691+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8de5140 0x55aec8de7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.691+0000 7fd8ee7fc700 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8de5140 0x55aec8de7530 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3483bc7c984a04 server_cookie=7415c179b04d1a96 in_seq=0 out_seq=0 2023-12-11T19:35:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.691+0000 7fd8d2ff5700 10 client.25675 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd8d2ff5700 1 -- 172.21.15.66:0/874193226 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aec8c93f70 con 0x55aec8de5140 2023-12-11T19:35:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd8d2ff5700 10 client.25675 handle_command_reply: tid=0 2023-12-11T19:35:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 client.25675 shutdown 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 msgr2=0x55aec8d87540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d87540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000c00 tx=0x7fd8e00b90d0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 shutdown_connections 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55aec8de5140 0x55aec8de7530 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aec8d7a720 0x55aec8d88970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aec8d78ed0 0x55aec8d87540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd8d2ff5700 0 client.25675 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 --2- 172.21.15.66:0/874193226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aec8d79830 0x55aec8d87a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 >> 172.21.15.66:0/874193226 conn(0x55aec8ca8780 msgr2=0x55aec8d82b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 shutdown_connections 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 1 -- 172.21.15.66:0/874193226 wait complete. 2023-12-11T19:35:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.702+0000 7fd9090ffb80 20 client.25675 trim_cache size 0 max 16384 2023-12-11T19:35:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:35:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:35:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:35:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd8d80a1480 msgr2=0x7fd8d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd8d80a1480 0x7fd8d80a3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80060b0 tx=0x7fd8e8007a80 comp rx=0 tx=0).stop 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 msgr2=0x7fd8f01996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f01996a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009fd0 tx=0x7fd8e000fa00 comp rx=0 tx=0).stop 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 shutdown_connections 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd8d80a1480 0x7fd8d80a3940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8f01030c0 0x7fd8f01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8f0101530 0x7fd8f01996a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 --2- 172.21.15.66:0/4236531440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8f0102720 0x7fd8f0199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 >> 172.21.15.66:0/4236531440 conn(0x7fd8f00fcaa0 msgr2=0x7fd8f0107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 shutdown_connections 2023-12-11T19:35:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:18.703+0000 7fd8f57ff700 1 -- 172.21.15.66:0/4236531440 wait complete. 2023-12-11T19:35:19.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:19 smithi070 ceph-mon[104044]: pgmap v412: 97 pgs: 97 active+clean; 981 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 54 MiB/s wr, 25 op/s 2023-12-11T19:35:20.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:19 smithi167 ceph-mon[102750]: pgmap v412: 97 pgs: 97 active+clean; 981 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 54 MiB/s wr, 25 op/s 2023-12-11T19:35:20.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:19 smithi066 ceph-mon[97920]: pgmap v412: 97 pgs: 97 active+clean; 981 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 54 MiB/s wr, 25 op/s 2023-12-11T19:35:21.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:21 smithi070 ceph-mon[104044]: pgmap v413: 97 pgs: 97 active+clean; 1.0 GiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 50 MiB/s wr, 28 op/s 2023-12-11T19:35:22.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:21 smithi167 ceph-mon[102750]: pgmap v413: 97 pgs: 97 active+clean; 1.0 GiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 50 MiB/s wr, 28 op/s 2023-12-11T19:35:22.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:21 smithi066 ceph-mon[97920]: pgmap v413: 97 pgs: 97 active+clean; 1.0 GiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 50 MiB/s wr, 28 op/s 2023-12-11T19:35:23.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:23 smithi070 ceph-mon[104044]: pgmap v414: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 39 MiB/s wr, 21 op/s 2023-12-11T19:35:24.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:23 smithi167 ceph-mon[102750]: pgmap v414: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 39 MiB/s wr, 21 op/s 2023-12-11T19:35:24.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:23 smithi066 ceph-mon[97920]: pgmap v414: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 39 MiB/s wr, 21 op/s 2023-12-11T19:35:24.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:35:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 -- 172.21.15.66:0/2520984706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941049a0 msgr2=0x7fd094104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 --2- 172.21.15.66:0/2520984706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941049a0 0x7fd094104e00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0880099c0 tx=0x7fd088009cd0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 -- 172.21.15.66:0/2520984706 shutdown_connections 2023-12-11T19:35:25.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 --2- 172.21.15.66:0/2520984706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd094105340 0x7fd09410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 --2- 172.21.15.66:0/2520984706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0941037b0 0x7fd094103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 --2- 172.21.15.66:0/2520984706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941049a0 0x7fd094104e00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 -- 172.21.15.66:0/2520984706 >> 172.21.15.66:0/2520984706 conn(0x7fd0940fed20 msgr2=0x7fd094101160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 -- 172.21.15.66:0/2520984706 shutdown_connections 2023-12-11T19:35:25.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.217+0000 7fd099fff700 1 -- 172.21.15.66:0/2520984706 wait complete. 2023-12-11T19:35:25.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.218+0000 7fd099fff700 1 Processor -- start 2023-12-11T19:35:25.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.218+0000 7fd099fff700 1 -- start start 2023-12-11T19:35:25.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd099fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 0x7fd094199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd099fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0941049a0 0x7fd094199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd099fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094105340 0x7fd0941a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd099fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941a1190 con 0x7fd0941037b0 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd099fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941a12d0 con 0x7fd0941049a0 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd099fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941a1460 con 0x7fd094105340 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 0x7fd094199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 0x7fd094199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51584/0 (socket says 172.21.15.66:51584) 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd0937fe700 1 -- 172.21.15.66:0/3729414297 learned_addr learned my addr 172.21.15.66:0/3729414297 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd092ffd700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0941049a0 0x7fd094199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.219+0000 7fd093fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094105340 0x7fd0941a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd0937fe700 1 -- 172.21.15.66:0/3729414297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094105340 msgr2=0x7fd0941a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd0937fe700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094105340 0x7fd0941a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd0937fe700 1 -- 172.21.15.66:0/3729414297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0941049a0 msgr2=0x7fd094199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd0937fe700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0941049a0 0x7fd094199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd0937fe700 1 -- 172.21.15.66:0/3729414297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880095f0 con 0x7fd0941037b0 2023-12-11T19:35:25.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd092ffd700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0941049a0 0x7fd094199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:25.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd0937fe700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 0x7fd094199660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd08400e970 tx=0x7fd08400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd090ff9700 1 -- 172.21.15.66:0/3729414297 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084009790 con 0x7fd0941037b0 2023-12-11T19:35:25.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd090ff9700 1 -- 172.21.15.66:0/3729414297 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd08400c7e0 con 0x7fd0941037b0 2023-12-11T19:35:25.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0941a1740 con 0x7fd0941037b0 2023-12-11T19:35:25.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941a1ca0 con 0x7fd0941037b0 2023-12-11T19:35:25.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.220+0000 7fd090ff9700 1 -- 172.21.15.66:0/3729414297 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084003bd0 con 0x7fd0941037b0 2023-12-11T19:35:25.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.225+0000 7fd090ff9700 1 -- 172.21.15.66:0/3729414297 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd084018430 con 0x7fd0941037b0 2023-12-11T19:35:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.225+0000 7fd0ad971b80 1 Processor -- start 2023-12-11T19:35:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.225+0000 7fd0ad971b80 1 -- start start 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c1407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c141640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c09bb50 con 0x55c59c1403d0 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c099470 con 0x55c59c140d30 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c094d60 con 0x55c59c141c20 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd093fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c1407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd093fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c1407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51586/0 (socket says 172.21.15.66:51586) 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.226+0000 7fd093fff700 1 -- 172.21.15.66:0/845665421 learned_addr learned my addr 172.21.15.66:0/845665421 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd0937fe700 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd092ffd700 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd090ff9700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd07c0a1500 0x7fd07c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 -- 172.21.15.66:0/845665421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 msgr2=0x55c59c1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 -- 172.21.15.66:0/845665421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 msgr2=0x55c59c141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 -- 172.21.15.66:0/845665421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c59bf11630 con 0x55c59c1403d0 2023-12-11T19:35:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd090ff9700 1 -- 172.21.15.66:0/3729414297 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd084014070 con 0x7fd0941037b0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd07c0a1500 0x7fd07c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.227+0000 7fd093fff700 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c1407f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd08c003f90 tx=0x7fd08c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db5f8a0b4d5a843 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.228+0000 7fd079ffb700 1 -- 172.21.15.66:0/845665421 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004870 con 0x55c59c1403d0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.228+0000 7fd079ffb700 1 -- 172.21.15.66:0/845665421 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd08c00f040 con 0x55c59c1403d0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.228+0000 7fd079ffb700 1 -- 172.21.15.66:0/845665421 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c015c70 con 0x55c59c1403d0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.228+0000 7fd0ad971b80 1 -- 172.21.15.66:0/845665421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 msgr2=0x55c59c1407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c1407f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd08c003f90 tx=0x7fd08c009c20 comp rx=0 tx=0).stop 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.228+0000 7fd093fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd07c0a1500 0x7fd07c0a39c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01c000 tx=0x7fd08c021420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 -- 172.21.15.66:0/845665421 shutdown_connections 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c1407f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/845665421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 -- 172.21.15.66:0/845665421 >> 172.21.15.66:0/845665421 conn(0x55c59c06f7e0 msgr2=0x55c59c13cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 -- 172.21.15.66:0/845665421 shutdown_connections 2023-12-11T19:35:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.229+0000 7fd0ad971b80 1 -- 172.21.15.66:0/845665421 wait complete. 2023-12-11T19:35:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 -- start start 2023-12-11T19:35:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 Processor -- start 2023-12-11T19:35:25.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c14a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c14a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c04e560 con 0x55c59c1403d0 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c04e200 con 0x55c59c140d30 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59bf13920 con 0x55c59c141c20 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.230+0000 7fd0ad971b80 10 client.0 _ll_register_callbacks cb 0x55c59bf79a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c14a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51600/0 (socket says 172.21.15.66:51600) 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 -- 172.21.15.66:0/700646396 learned_addr learned my addr 172.21.15.66:0/700646396 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd092ffd700 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 -- 172.21.15.66:0/700646396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 msgr2=0x55c59c1516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 -- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 msgr2=0x55c59c14a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c14a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.231+0000 7fd0937fe700 1 -- 172.21.15.66:0/700646396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c003bf0 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd0937fe700 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c14a120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0840a1f40 tx=0x7fd084096570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db32fd77f4751e05 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd072ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd072ffd700 1 -- 172.21.15.66:0/700646396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084099b90 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd072ffd700 1 -- 172.21.15.66:0/700646396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0840a2470 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c59c133b40 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd072ffd700 1 -- 172.21.15.66:0/700646396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd084099720 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd0ad971b80 10 client.15960 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c59bebdde0 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd072ffd700 1 -- 172.21.15.66:0/700646396 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c59c133b40 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.232+0000 7fd072ffd700 1 -- 172.21.15.66:0/700646396 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd0840a1fe0 con 0x55c59c1403d0 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd0ad971b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd0ad971b80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd0ad971b80 20 client.15960 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c155f40 0x55c59c158360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd0ad971b80 4 client.15960 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c59c10ff40 con 0x55c59c155f40 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.233+0000 7fd092ffd700 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c155f40 0x55c59c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.234+0000 7fd092ffd700 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c155f40 0x55c59c158360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad9c9b9e52e9d697 server_cookie=1c270d821f340dc1 in_seq=0 out_seq=0 2023-12-11T19:35:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.234+0000 7fd072ffd700 10 client.15960 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.234+0000 7fd072ffd700 1 -- 172.21.15.66:0/700646396 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c59c10ff40 con 0x55c59c155f40 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.234+0000 7fd072ffd700 10 client.15960 handle_command_reply: tid=0 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.234+0000 7fd0ad971b80 1 client.15960 shutdown 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 msgr2=0x55c59c14a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c14a120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0840a1f40 tx=0x7fd084096570 comp rx=0 tx=0).stop 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 shutdown_connections 2023-12-11T19:35:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c155f40 0x55c59c158360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c141c20 0x55c59c1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c14a120 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd072ffd700 0 client.15960 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/700646396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c140d30 0x55c59c14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 >> 172.21.15.66:0/700646396 conn(0x55c59c06f7e0 msgr2=0x55c59c13ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 shutdown_connections 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 1 -- 172.21.15.66:0/700646396 wait complete. 2023-12-11T19:35:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.235+0000 7fd0ad971b80 20 client.15960 trim_cache size 0 max 16384 2023-12-11T19:35:25.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.254+0000 7fd0ad971b80 1 Processor -- start 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 -- start start 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c142000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c140d30 0x55c59c151630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c13c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c14b180 con 0x55c59c1403d0 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c153400 con 0x55c59c141c20 2023-12-11T19:35:25.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c137b80 con 0x55c59c140d30 2023-12-11T19:35:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c142000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51602/0 (socket says 172.21.15.66:51602) 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd0937fe700 1 -- 172.21.15.66:0/3184096127 learned_addr learned my addr 172.21.15.66:0/3184096127 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd092ffd700 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c140d30 0x55c59c151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.255+0000 7fd093fff700 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c13c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd0937fe700 1 -- 172.21.15.66:0/3184096127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c140d30 msgr2=0x55c59c151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd0937fe700 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c140d30 0x55c59c151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd0937fe700 1 -- 172.21.15.66:0/3184096127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 msgr2=0x55c59c13c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd0937fe700 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c13c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd0937fe700 1 -- 172.21.15.66:0/3184096127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c59bea6c60 con 0x55c59c1403d0 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd092ffd700 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c140d30 0x55c59c151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd0937fe700 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c142000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd084096730 tx=0x7fd0840987e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2063a66a5e72e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd07a7fc700 1 -- 172.21.15.66:0/3184096127 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084099b90 con 0x55c59c1403d0 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd07a7fc700 1 -- 172.21.15.66:0/3184096127 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0840a2470 con 0x55c59c1403d0 2023-12-11T19:35:25.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.256+0000 7fd07a7fc700 1 -- 172.21.15.66:0/3184096127 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840963d0 con 0x55c59c1403d0 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 -- 172.21.15.66:0/3184096127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 msgr2=0x55c59c142000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c142000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd084096730 tx=0x7fd0840987e0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 -- 172.21.15.66:0/3184096127 shutdown_connections 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c13c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c1403d0 0x55c59c142000 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/3184096127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c140d30 0x55c59c151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 -- 172.21.15.66:0/3184096127 >> 172.21.15.66:0/3184096127 conn(0x55c59c06f7e0 msgr2=0x55c59c144af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 -- 172.21.15.66:0/3184096127 shutdown_connections 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.257+0000 7fd0ad971b80 1 -- 172.21.15.66:0/3184096127 wait complete. 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.258+0000 7fd0ad971b80 1 -- start start 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.258+0000 7fd0ad971b80 1 Processor -- start 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c1403d0 0x55c59c14d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 0x55c59c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c150750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c150c90 con 0x55c59c140d30 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c150e00 con 0x55c59c141c20 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59c150fc0 con 0x55c59c1403d0 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0ad971b80 10 client.0 _ll_register_callbacks cb 0x55c59bf79a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd092ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 0x55c59c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd092ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 0x55c59c14d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51614/0 (socket says 172.21.15.66:51614) 2023-12-11T19:35:25.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd092ffd700 1 -- 172.21.15.66:0/780219317 learned_addr learned my addr 172.21.15.66:0/780219317 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:25.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd0937fe700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c1403d0 0x55c59c14d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.259+0000 7fd093fff700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd0937fe700 1 -- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 msgr2=0x55c59c150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd0937fe700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd0937fe700 1 -- 172.21.15.66:0/780219317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 msgr2=0x55c59c14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd0937fe700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 0x55c59c14d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd0937fe700 1 -- 172.21.15.66:0/780219317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09410c540 con 0x55c59c1403d0 2023-12-11T19:35:25.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd092ffd700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 0x55c59c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd0937fe700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c1403d0 0x55c59c14d1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd084099ad0 tx=0x7fd0840d0640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2dbad77fd498fd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.260+0000 7fd072ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd072ffd700 1 -- 172.21.15.66:0/780219317 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084099b90 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd08c00f040 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd072ffd700 1 -- 172.21.15.66:0/780219317 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0840b82a0 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd072ffd700 1 -- 172.21.15.66:0/780219317 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd084099720 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd0ad971b80 10 client.25690 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c59bebdde0 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd072ffd700 1 -- 172.21.15.66:0/780219317 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd08c00f040 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.261+0000 7fd072ffd700 1 -- 172.21.15.66:0/780219317 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd084099310 con 0x55c59c1403d0 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd0ad971b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd0ad971b80 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd0ad971b80 20 client.25690 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c1acc70 0x55c59c1af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd0ad971b80 4 client.25690 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:35:25.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c59c05afd0 con 0x55c59c1acc70 2023-12-11T19:35:25.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.262+0000 7fd092ffd700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c1acc70 0x55c59c1af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd092ffd700 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c1acc70 0x55c59c1af060 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38be2101b1da4e15 server_cookie=7819280396c11ef2 in_seq=0 out_seq=0 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd072ffd700 10 client.25690 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd072ffd700 1 -- 172.21.15.66:0/780219317 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c59c05afd0 con 0x55c59c1acc70 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd072ffd700 10 client.25690 handle_command_reply: tid=0 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd0ad971b80 1 client.25690 shutdown 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c1403d0 msgr2=0x55c59c14d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.263+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c1403d0 0x55c59c14d1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd084099ad0 tx=0x7fd0840d0640 comp rx=0 tx=0).stop 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 shutdown_connections 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55c59c1acc70 0x55c59c1af060 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c59c141c20 0x55c59c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c59c1403d0 0x55c59c14d1f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd072ffd700 0 client.25690 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 --2- 172.21.15.66:0/780219317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c59c140d30 0x55c59c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 >> 172.21.15.66:0/780219317 conn(0x55c59c06f7e0 msgr2=0x55c59c13c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 shutdown_connections 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 1 -- 172.21.15.66:0/780219317 wait complete. 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.264+0000 7fd0ad971b80 20 client.25690 trim_cache size 0 max 16384 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-12-11T19:35:25.267 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd07c0a1500 msgr2=0x7fd07c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd07c0a1500 0x7fd07c0a39c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd08c01c000 tx=0x7fd08c021420 comp rx=0 tx=0).stop 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 msgr2=0x7fd094199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 0x7fd094199660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd08400e970 tx=0x7fd08400ec80 comp rx=0 tx=0).stop 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 shutdown_connections 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fd07c0a1500 0x7fd07c0a39c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd094105340 0x7fd0941a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd0941037b0 0x7fd094199660 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 --2- 172.21.15.66:0/3729414297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0941049a0 0x7fd094199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 >> 172.21.15.66:0/3729414297 conn(0x7fd0940fed20 msgr2=0x7fd094107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 shutdown_connections 2023-12-11T19:35:25.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.265+0000 7fd099fff700 1 -- 172.21.15.66:0/3729414297 wait complete. 2023-12-11T19:35:25.289 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-11T19:35:25.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:35:25.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 -- 172.21.15.66:0/3854528497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc073fd0 msgr2=0x7f2dcc074450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/3854528497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc073fd0 0x7f2dcc074450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009990 tx=0x7f2db8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 -- 172.21.15.66:0/3854528497 shutdown_connections 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/3854528497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/3854528497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc109380 0x7f2dcc10b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/3854528497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc073fd0 0x7f2dcc074450 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 -- 172.21.15.66:0/3854528497 >> 172.21.15.66:0/3854528497 conn(0x7f2dcc0fca20 msgr2=0x7f2dcc0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 -- 172.21.15.66:0/3854528497 shutdown_connections 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.546+0000 7f2dd2fff700 1 -- 172.21.15.66:0/3854528497 wait complete. 2023-12-11T19:35:25.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.548+0000 7f2dd2fff700 1 Processor -- start 2023-12-11T19:35:25.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.548+0000 7f2dd2fff700 1 -- start start 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.548+0000 7f2dd2fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dcc073fd0 0x7f2dcc199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.548+0000 7f2dd2fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.548+0000 7f2dd2fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc109380 0x7f2dcc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dd2fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1a1060 con 0x7f2dcc109380 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dd2fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1a11a0 con 0x7f2dcc073fd0 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dd2fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1a1330 con 0x7f2dcc0755b0 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37098/0 (socket says 172.21.15.66:37098) 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 -- 172.21.15.66:0/1328876209 learned_addr learned my addr 172.21.15.66:0/1328876209 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dd159c700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc109380 0x7f2dcc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dd0d9b700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dcc073fd0 0x7f2dcc199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 -- 172.21.15.66:0/1328876209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dcc073fd0 msgr2=0x7f2dcc199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dcc073fd0 0x7f2dcc199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 -- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc109380 msgr2=0x7f2dcc1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc109380 0x7f2dcc1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.549+0000 7f2dcbfff700 1 -- 172.21.15.66:0/1328876209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc0009520 con 0x7f2dcc0755b0 2023-12-11T19:35:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.550+0000 7f2dcbfff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc199a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2db8005740 tx=0x7f2db8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.550+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db801d070 con 0x7f2dcc0755b0 2023-12-11T19:35:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.550+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2db800be70 con 0x7f2dcc0755b0 2023-12-11T19:35:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.550+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db80095f0 con 0x7f2dcc0755b0 2023-12-11T19:35:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.550+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8021410 con 0x7f2dcc0755b0 2023-12-11T19:35:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.550+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc1a1840 con 0x7f2dcc0755b0 2023-12-11T19:35:25.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.551+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc108190 con 0x7f2dcc0755b0 2023-12-11T19:35:25.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.553+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2db800f640 con 0x7f2dcc0755b0 2023-12-11T19:35:25.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.555+0000 7f2dc9ffb700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2dbc0a1480 0x7f2dbc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:25.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.556+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2db80ce060 con 0x7f2dcc0755b0 2023-12-11T19:35:25.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.556+0000 7f2dd0d9b700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2dbc0a1480 0x7f2dbc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:25.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.556+0000 7f2dd0d9b700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2dbc0a1480 0x7f2dbc0a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009df0 tx=0x7f2dc0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:25.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.559+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2db8097710 con 0x7f2dcc0755b0 2023-12-11T19:35:25.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:25 smithi066 ceph-mon[97920]: pgmap v415: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-12-11T19:35:25.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:25 smithi070 ceph-mon[104044]: pgmap v415: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-12-11T19:35:25.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.888+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dcc068230 con 0x7f2dcc0755b0 2023-12-11T19:35:25.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.889+0000 7f2dc9ffb700 1 -- 172.21.15.66:0/1328876209 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11363 (secure 0 0 0) 0x7f2db809ad30 con 0x7f2dcc0755b0 2023-12-11T19:35:25.890 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:35:25.890 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:35:10.880436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2dbc0a1480 msgr2=0x7f2dbc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2dbc0a1480 0x7f2dbc0a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009df0 tx=0x7f2dc0009260 comp rx=0 tx=0).stop 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 msgr2=0x7f2dcc199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc199a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2db8005740 tx=0x7f2db8009580 comp rx=0 tx=0).stop 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 shutdown_connections 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2dbc0a1480 0x7f2dbc0a3940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dcc073fd0 0x7f2dcc199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dcc109380 0x7f2dcc1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 --2- 172.21.15.66:0/1328876209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dcc0755b0 0x7f2dcc199a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 >> 172.21.15.66:0/1328876209 conn(0x7f2dcc0fca20 msgr2=0x7f2dcc1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 shutdown_connections 2023-12-11T19:35:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:25.892+0000 7f2dd2fff700 1 -- 172.21.15.66:0/1328876209 wait complete. 2023-12-11T19:35:25.894 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 28 2023-12-11T19:35:25.913 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-11T19:35:26.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:25 smithi167 ceph-mon[102750]: pgmap v415: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-12-11T19:35:26.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.154+0000 7f7199f9d700 1 -- 172.21.15.66:0/688821823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71941047e0 msgr2=0x7f7194106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.154+0000 7f7199f9d700 1 --2- 172.21.15.66:0/688821823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71941047e0 0x7f7194106bd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7180009990 tx=0x7f7180009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.154+0000 7f7199f9d700 1 -- 172.21.15.66:0/688821823 shutdown_connections 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.154+0000 7f7199f9d700 1 --2- 172.21.15.66:0/688821823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 0x7f7194109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.154+0000 7f7199f9d700 1 --2- 172.21.15.66:0/688821823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7194101eb0 0x7f71941042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.155+0000 7f7199f9d700 1 --2- 172.21.15.66:0/688821823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71941047e0 0x7f7194106bd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.155+0000 7f7199f9d700 1 -- 172.21.15.66:0/688821823 >> 172.21.15.66:0/688821823 conn(0x7f71940fb820 msgr2=0x7f71940fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.155+0000 7f7199f9d700 1 -- 172.21.15.66:0/688821823 shutdown_connections 2023-12-11T19:35:26.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.155+0000 7f7199f9d700 1 -- 172.21.15.66:0/688821823 wait complete. 2023-12-11T19:35:26.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.156+0000 7f7199f9d700 1 Processor -- start 2023-12-11T19:35:26.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.156+0000 7f7199f9d700 1 -- start start 2023-12-11T19:35:26.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7199f9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 0x7f7194197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7199f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71941047e0 0x7f71941977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7199f9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 0x7f719419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7199f9d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719419edb0 con 0x7f7194101eb0 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7199f9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719419eef0 con 0x7f7194107110 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7199f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719419f080 con 0x7f71941047e0 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 0x7f7194197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 0x7f7194197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51672/0 (socket says 172.21.15.66:51672) 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7198f9b700 1 -- 172.21.15.66:0/1460323169 learned_addr learned my addr 172.21.15.66:0/1460323169 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f7193fff700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71941047e0 0x7f71941977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.157+0000 7f719979c700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 0x7f719419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7198f9b700 1 -- 172.21.15.66:0/1460323169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71941047e0 msgr2=0x7f71941977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7198f9b700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71941047e0 0x7f71941977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7198f9b700 1 -- 172.21.15.66:0/1460323169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 msgr2=0x7f719419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7198f9b700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 0x7f719419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7198f9b700 1 -- 172.21.15.66:0/1460323169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800095f0 con 0x7f7194101eb0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f719979c700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 0x7f719419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7198f9b700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 0x7f7194197280 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f718800d980 tx=0x7f718800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7191ffb700 1 -- 172.21.15.66:0/1460323169 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188004220 con 0x7f7194101eb0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7191ffb700 1 -- 172.21.15.66:0/1460323169 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7188007640 con 0x7f7194101eb0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719419f360 con 0x7f7194101eb0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7191ffb700 1 -- 172.21.15.66:0/1460323169 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71880055e0 con 0x7f7194101eb0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.158+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719419f8c0 con 0x7f7194101eb0 2023-12-11T19:35:26.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.162+0000 7f7191ffb700 1 -- 172.21.15.66:0/1460323169 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7188004750 con 0x7f7194101eb0 2023-12-11T19:35:26.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.164+0000 7f7191ffb700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f71840a1500 0x7f71840a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.164+0000 7f71aea3db80 1 Processor -- start 2023-12-11T19:35:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f7191ffb700 1 -- 172.21.15.66:0/1460323169 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71880cc8c0 con 0x7f7194101eb0 2023-12-11T19:35:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 -- start start 2023-12-11T19:35:26.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f7193fff700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f71840a1500 0x7f71840a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb1683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb166c40 0x5623eb1670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb16eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb0bdb50 con 0x5623eb167f80 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb0bb470 con 0x5623eb167600 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.165+0000 7f71aea3db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb0b6d60 con 0x5623eb166c40 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7193fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb166c40 0x5623eb1670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb1683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb1683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51682/0 (socket says 172.21.15.66:51682) 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7193fff700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f71840a1500 0x7f71840a39c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7180005910 tx=0x7f718000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 -- 172.21.15.66:0/3493420957 learned_addr learned my addr 172.21.15.66:0/3493420957 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f719979c700 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb16eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 -- 172.21.15.66:0/3493420957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb166c40 msgr2=0x5623eb1670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb166c40 0x5623eb1670c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 -- 172.21.15.66:0/3493420957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 msgr2=0x5623eb16eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb16eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 -- 172.21.15.66:0/3493420957 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623eaf33630 con 0x5623eb167f80 2023-12-11T19:35:26.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.166+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb1683a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f71880a1a40 tx=0x7f7188005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbce237ec5dd35b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.167+0000 7f717effd700 1 -- 172.21.15.66:0/3493420957 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718809a2e0 con 0x5623eb167f80 2023-12-11T19:35:26.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.167+0000 7f717effd700 1 -- 172.21.15.66:0/3493420957 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7188099be0 con 0x5623eb167f80 2023-12-11T19:35:26.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.167+0000 7f717effd700 1 -- 172.21.15.66:0/3493420957 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718809a0a0 con 0x5623eb167f80 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.167+0000 7f71aea3db80 1 -- 172.21.15.66:0/3493420957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 msgr2=0x5623eb1683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.167+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb1683a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f71880a1a40 tx=0x7f7188005810 comp rx=0 tx=0).stop 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 -- 172.21.15.66:0/3493420957 shutdown_connections 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb16eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb1683a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3493420957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb166c40 0x5623eb1670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 -- 172.21.15.66:0/3493420957 >> 172.21.15.66:0/3493420957 conn(0x5623eb0917e0 msgr2=0x5623eb15d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 -- 172.21.15.66:0/3493420957 shutdown_connections 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 -- 172.21.15.66:0/3493420957 wait complete. 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.168+0000 7f71aea3db80 1 -- start start 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 Processor -- start 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb170880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb170dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167f80 0x5623eb177e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb070560 con 0x5623eb166c40 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb070200 con 0x5623eb167600 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eaf35920 con 0x5623eb167f80 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f71aea3db80 10 client.0 _ll_register_callbacks cb 0x5623eaf9ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb170880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f7193fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb170dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f7193fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb170dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36902/0 (socket says 172.21.15.66:36902) 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.169+0000 7f7193fff700 1 -- 172.21.15.66:0/3194659097 learned_addr learned my addr 172.21.15.66:0/3194659097 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f719979c700 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167f80 0x5623eb177e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f7198f9b700 1 -- 172.21.15.66:0/3194659097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167f80 msgr2=0x5623eb177e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167f80 0x5623eb177e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f7198f9b700 1 -- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 msgr2=0x5623eb170dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb170dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f7198f9b700 1 -- 172.21.15.66:0/3194659097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719419f8c0 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.170+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb170880 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f71880960e0 tx=0x7f7188004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfb1e146afbf24d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f7173fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f7173fff700 1 -- 172.21.15.66:0/3194659097 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718809a2e0 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f7173fff700 1 -- 172.21.15.66:0/3194659097 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f718809dd30 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623eb1559e0 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f7173fff700 1 -- 172.21.15.66:0/3194659097 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71880b2210 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f71aea3db80 10 client.15984 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5623eaedfde0 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f7173fff700 1 -- 172.21.15.66:0/3194659097 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5623eb1559e0 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.171+0000 7f7173fff700 1 -- 172.21.15.66:0/3194659097 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f71880a1b50 con 0x5623eb166c40 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f71aea3db80 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f71aea3db80 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f71aea3db80 20 client.15984 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:26.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb17c1a0 0x5623eb17e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f71aea3db80 4 client.15984 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623eb131f40 con 0x5623eb17c1a0 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.172+0000 7f7193fff700 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb17c1a0 0x5623eb17e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.173+0000 7f7193fff700 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb17c1a0 0x5623eb17e590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7911adfd0b268690 server_cookie=ae222ff0d46cee79 in_seq=0 out_seq=0 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.173+0000 7f7173fff700 10 client.15984 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.173+0000 7f7173fff700 1 -- 172.21.15.66:0/3194659097 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623eb131f40 con 0x5623eb17c1a0 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.173+0000 7f7173fff700 10 client.15984 handle_command_reply: tid=0 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.173+0000 7f71aea3db80 1 client.15984 shutdown 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 msgr2=0x5623eb170880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb170880 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f71880960e0 tx=0x7f7188004570 comp rx=0 tx=0).stop 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 shutdown_connections 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb17c1a0 0x5623eb17e590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167f80 0x5623eb177e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb170880 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f7173fff700 0 client.15984 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3194659097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167600 0x5623eb170dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 >> 172.21.15.66:0/3194659097 conn(0x5623eb0917e0 msgr2=0x5623eb161820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 shutdown_connections 2023-12-11T19:35:26.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 1 -- 172.21.15.66:0/3194659097 wait complete. 2023-12-11T19:35:26.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.174+0000 7f71aea3db80 20 client.15984 trim_cache size 0 max 16384 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.193+0000 7f71aea3db80 1 Processor -- start 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.193+0000 7f71aea3db80 1 -- start start 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 0x5623eb168360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb177dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb16fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f71aea3db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb178f80 con 0x5623eb167f80 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f71aea3db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb15e750 con 0x5623eb166c40 2023-12-11T19:35:26.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f71aea3db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb1703f0 con 0x5623eb167600 2023-12-11T19:35:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 0x5623eb168360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f719979c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb16fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 0x5623eb168360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:36904/0 (socket says 172.21.15.66:36904) 2023-12-11T19:35:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f7198f9b700 1 -- 172.21.15.66:0/3679722685 learned_addr learned my addr 172.21.15.66:0/3679722685 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f7193fff700 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb177dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f7198f9b700 1 -- 172.21.15.66:0/3679722685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 msgr2=0x5623eb177dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.194+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb177dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f7198f9b700 1 -- 172.21.15.66:0/3679722685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 msgr2=0x5623eb16fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb16fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f7198f9b700 1 -- 172.21.15.66:0/3679722685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623eaec8c60 con 0x5623eb166c40 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 0x5623eb168360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f718800b540 tx=0x7f7188095f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe11013e24a4aba server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f717f7fe700 1 -- 172.21.15.66:0/3679722685 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718809a2e0 con 0x5623eb166c40 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f717f7fe700 1 -- 172.21.15.66:0/3679722685 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718809dd30 con 0x5623eb166c40 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.195+0000 7f717f7fe700 1 -- 172.21.15.66:0/3679722685 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188099e40 con 0x5623eb166c40 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 -- 172.21.15.66:0/3679722685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 msgr2=0x5623eb168360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 0x5623eb168360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f718800b540 tx=0x7f7188095f20 comp rx=0 tx=0).stop 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 -- 172.21.15.66:0/3679722685 shutdown_connections 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb166c40 0x5623eb168360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb167f80 0x5623eb16fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3679722685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb177dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 -- 172.21.15.66:0/3679722685 >> 172.21.15.66:0/3679722685 conn(0x5623eb0917e0 msgr2=0x5623eb15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 -- 172.21.15.66:0/3679722685 shutdown_connections 2023-12-11T19:35:26.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.196+0000 7f71aea3db80 1 -- 172.21.15.66:0/3679722685 wait complete. 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.197+0000 7f71aea3db80 1 -- start start 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.197+0000 7f71aea3db80 1 Processor -- start 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb175360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb1758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167f80 0x5623eb176810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb176d50 con 0x5623eb166c40 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb176ec0 con 0x5623eb167f80 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623eb177080 con 0x5623eb167600 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f71aea3db80 10 client.0 _ll_register_callbacks cb 0x5623eaf9ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb175360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f7193fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb1758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f7193fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb1758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37168/0 (socket says 172.21.15.66:37168) 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f7193fff700 1 -- 172.21.15.66:0/3026498644 learned_addr learned my addr 172.21.15.66:0/3026498644 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.198+0000 7f7198f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb175360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51722/0 (socket says 172.21.15.66:51722) 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f719979c700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167f80 0x5623eb176810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7193fff700 1 -- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167f80 msgr2=0x5623eb176810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7193fff700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167f80 0x5623eb176810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7193fff700 1 -- 172.21.15.66:0/3026498644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 msgr2=0x5623eb175360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7193fff700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb175360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7193fff700 1 -- 172.21.15.66:0/3026498644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623eaedfde0 con 0x5623eb167600 2023-12-11T19:35:26.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb175360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7193fff700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb1758a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71800098e0 tx=0x7f718001a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb227a9cbcdf0cd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.199+0000 7f7173fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:35:26.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f7173fff700 1 -- 172.21.15.66:0/3026498644 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000b710 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f7173fff700 1 -- 172.21.15.66:0/3026498644 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f718000b870 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7188099be0 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f7173fff700 1 -- 172.21.15.66:0/3026498644 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f718000ba30 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f71aea3db80 10 client.25714 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5623eaf06250 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.200+0000 7f7173fff700 1 -- 172.21.15.66:0/3026498644 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7188099be0 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f7173fff700 1 -- 172.21.15.66:0/3026498644 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7180035710 con 0x5623eb167600 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f71aea3db80 10 client.25714 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f71aea3db80 10 client.25714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f71aea3db80 20 client.25714 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb1d2d60 0x5623eb1d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f71aea3db80 4 client.25714 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5623eb07cfd0 con 0x5623eb1d2d60 2023-12-11T19:35:26.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.201+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb1d2d60 0x5623eb1d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.202+0000 7f7173fff700 10 client.25714 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.202+0000 7f7198f9b700 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb1d2d60 0x5623eb1d5180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed09ab1b775e4dba server_cookie=fee9c6a3d03d2f05 in_seq=0 out_seq=0 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.202+0000 7f7173fff700 1 -- 172.21.15.66:0/3026498644 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5623eb07cfd0 con 0x5623eb1d2d60 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.202+0000 7f7173fff700 10 client.25714 handle_command_reply: tid=0 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.202+0000 7f71aea3db80 1 client.25714 shutdown 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 msgr2=0x5623eb1758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb1758a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71800098e0 tx=0x7f718001a050 comp rx=0 tx=0).stop 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 shutdown_connections 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x5623eb1d2d60 0x5623eb1d5180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5623eb167f80 0x5623eb176810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623eb166c40 0x5623eb175360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 --2- 172.21.15.66:0/3026498644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623eb167600 0x5623eb1758a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 >> 172.21.15.66:0/3026498644 conn(0x5623eb0917e0 msgr2=0x5623eb16ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f7173fff700 0 client.25714 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:35:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 shutdown_connections 2023-12-11T19:35:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 1 -- 172.21.15.66:0/3026498644 wait complete. 2023-12-11T19:35:26.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.203+0000 7f71aea3db80 20 client.25714 trim_cache size 0 max 16384 2023-12-11T19:35:26.208 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f71840a1500 msgr2=0x7f71840a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f71840a1500 0x7f71840a39c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7180005910 tx=0x7f718000b610 comp rx=0 tx=0).stop 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 msgr2=0x7f7194197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 0x7f7194197280 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f718800d980 tx=0x7f718800dc90 comp rx=0 tx=0).stop 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 shutdown_connections 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f71840a1500 0x7f71840a39c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7194107110 0x7f719419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7194101eb0 0x7f7194197280 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 --2- 172.21.15.66:0/1460323169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71941047e0 0x7f71941977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 >> 172.21.15.66:0/1460323169 conn(0x7f71940fb820 msgr2=0x7f7194103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 shutdown_connections 2023-12-11T19:35:26.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.204+0000 7f7199f9d700 1 -- 172.21.15.66:0/1460323169 wait complete. 2023-12-11T19:35:26.226 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-11T19:35:26.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 -- 172.21.15.66:0/3718605561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1037b0 msgr2=0x7f612c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 --2- 172.21.15.66:0/3718605561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1037b0 0x7f612c103bd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f611c0099c0 tx=0x7f611c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 -- 172.21.15.66:0/3718605561 shutdown_connections 2023-12-11T19:35:26.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 --2- 172.21.15.66:0/3718605561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c105340 0x7f612c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 --2- 172.21.15.66:0/3718605561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1037b0 0x7f612c103bd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 --2- 172.21.15.66:0/3718605561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f612c1049a0 0x7f612c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 -- 172.21.15.66:0/3718605561 >> 172.21.15.66:0/3718605561 conn(0x7f612c0fed20 msgr2=0x7f612c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 -- 172.21.15.66:0/3718605561 shutdown_connections 2023-12-11T19:35:26.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.468+0000 7f6131fff700 1 -- 172.21.15.66:0/3718605561 wait complete. 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.469+0000 7f6131fff700 1 Processor -- start 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.469+0000 7f6131fff700 1 -- start start 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f6131fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c1037b0 0x7f612c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f6131fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 0x7f612c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f6131fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f612c105340 0x7f612c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f6131fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1a11d0 con 0x7f612c1049a0 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f6131fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1a1310 con 0x7f612c1037b0 2023-12-11T19:35:26.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f6131fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1a14a0 con 0x7f612c105340 2023-12-11T19:35:26.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f612affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 0x7f612c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f612affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 0x7f612c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51738/0 (socket says 172.21.15.66:51738) 2023-12-11T19:35:26.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f612affd700 1 -- 172.21.15.66:0/64701654 learned_addr learned my addr 172.21.15.66:0/64701654 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.470+0000 7f612bfff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f612c105340 0x7f612c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f612affd700 1 -- 172.21.15.66:0/64701654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f612c105340 msgr2=0x7f612c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f612affd700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f612c105340 0x7f612c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f612affd700 1 -- 172.21.15.66:0/64701654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c1037b0 msgr2=0x7f612c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f612affd700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c1037b0 0x7f612c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f612affd700 1 -- 172.21.15.66:0/64701654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c0095f0 con 0x7f612c1049a0 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f612affd700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 0x7f612c199be0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f612000e9a0 tx=0x7f612000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.471+0000 7f6128ff9700 1 -- 172.21.15.66:0/64701654 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120009790 con 0x7f612c1049a0 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.472+0000 7f6128ff9700 1 -- 172.21.15.66:0/64701654 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f612000cbd0 con 0x7f612c1049a0 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.472+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c1a1780 con 0x7f612c1049a0 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.472+0000 7f6128ff9700 1 -- 172.21.15.66:0/64701654 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120018680 con 0x7f612c1049a0 2023-12-11T19:35:26.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.472+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c1a1c90 con 0x7f612c1049a0 2023-12-11T19:35:26.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.476+0000 7f6128ff9700 1 -- 172.21.15.66:0/64701654 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f612000cd40 con 0x7f612c1049a0 2023-12-11T19:35:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 Processor -- start 2023-12-11T19:35:26.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 -- start start 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6b3d0 0x55ba7df6b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df6c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6cc20 0x55ba7df6d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f6128ff9700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61140a1480 0x7f61140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7dec6b50 con 0x55ba7df6bd30 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7dec4470 con 0x55ba7df6b3d0 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.478+0000 7f61459dfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7debfd60 con 0x55ba7df6cc20 2023-12-11T19:35:26.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f6128ff9700 1 -- 172.21.15.66:0/64701654 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6120014070 con 0x7f612c1049a0 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df6c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df6c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51750/0 (socket says 172.21.15.66:51750) 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6cc20 0x55ba7df6d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 -- 172.21.15.66:0/2231235450 learned_addr learned my addr 172.21.15.66:0/2231235450 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612bfff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61140a1480 0x7f61140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612bfff700 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6b3d0 0x55ba7df6b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 -- 172.21.15.66:0/2231235450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6cc20 msgr2=0x55ba7df6d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6cc20 0x55ba7df6d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 -- 172.21.15.66:0/2231235450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6b3d0 msgr2=0x55ba7df6b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6b3d0 0x55ba7df6b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612b7fe700 1 -- 172.21.15.66:0/2231235450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba7dd3c630 con 0x55ba7df6bd30 2023-12-11T19:35:26.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.479+0000 7f612bfff700 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6b3d0 0x55ba7df6b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:26.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.480+0000 7f612b7fe700 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df6c640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f611c00bf10 tx=0x7f611c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfd6cda99f64b31 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.480+0000 7f612bfff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61140a1480 0x7f61140a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6124009fd0 tx=0x7f6124009390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.480+0000 7f6111ffb700 1 -- 172.21.15.66:0/2231235450 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c01d070 con 0x55ba7df6bd30 2023-12-11T19:35:26.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.480+0000 7f6111ffb700 1 -- 172.21.15.66:0/2231235450 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f611c00bc20 con 0x55ba7df6bd30 2023-12-11T19:35:26.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.480+0000 7f6111ffb700 1 -- 172.21.15.66:0/2231235450 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c004970 con 0x55ba7df6bd30 2023-12-11T19:35:26.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 -- 172.21.15.66:0/2231235450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 msgr2=0x55ba7df6c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df6c640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f611c00bf10 tx=0x7f611c000bc0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 -- 172.21.15.66:0/2231235450 shutdown_connections 2023-12-11T19:35:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6b3d0 0x55ba7df6b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6cc20 0x55ba7df6d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 --2- 172.21.15.66:0/2231235450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df6c640 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 -- 172.21.15.66:0/2231235450 >> 172.21.15.66:0/2231235450 conn(0x55ba7de9a7e0 msgr2=0x55ba7df67d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 -- 172.21.15.66:0/2231235450 shutdown_connections 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.481+0000 7f61459dfb80 1 -- 172.21.15.66:0/2231235450 wait complete. 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 -- start start 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 Processor -- start 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df75680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7de79560 con 0x55ba7df6bd30 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7de79200 con 0x55ba7df6cc20 2023-12-11T19:35:26.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.482+0000 7f61459dfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7dd3e920 con 0x55ba7df6b3d0 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f61459dfb80 10 client.0 _ll_register_callbacks cb 0x55ba7dda4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df75680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df75140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37202/0 (socket says 172.21.15.66:37202) 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612b7fe700 1 -- 172.21.15.66:0/3771303662 learned_addr learned my addr 172.21.15.66:0/3771303662 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612bfff700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612affd700 1 -- 172.21.15.66:0/3771303662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 msgr2=0x55ba7df75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612affd700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df75140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612affd700 1 -- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 msgr2=0x55ba7df7c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612affd700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.483+0000 7f612affd700 1 -- 172.21.15.66:0/3771303662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c10c540 con 0x55ba7df6bd30 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f612b7fe700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f612affd700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df75680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f612000c5b0 tx=0x7f61200a2480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d7d312ff670dcf server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f610affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f610affd700 1 -- 172.21.15.66:0/3771303662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612009a9a0 con 0x55ba7df6bd30 2023-12-11T19:35:26.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f610affd700 1 -- 172.21.15.66:0/3771303662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f612009a2a0 con 0x55ba7df6bd30 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba7df5eb40 con 0x55ba7df6bd30 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f610affd700 1 -- 172.21.15.66:0/3771303662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f612009e3f0 con 0x55ba7df6bd30 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f61459dfb80 10 client.16002 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.484+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ba7dce8de0 con 0x55ba7df6bd30 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f610affd700 1 -- 172.21.15.66:0/3771303662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ba7df5eb40 con 0x55ba7df6bd30 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f610affd700 1 -- 172.21.15.66:0/3771303662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6120065df0 con 0x55ba7df6bd30 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f61459dfb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:26.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f61459dfb80 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:35:26.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f61459dfb80 20 client.16002 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:26.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f61459dfb80 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7df80e70 0x55ba7df83290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f61459dfb80 4 client.16002 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:26.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.485+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba7df3af40 con 0x55ba7df80e70 2023-12-11T19:35:26.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.486+0000 7f612b7fe700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7df80e70 0x55ba7df83290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.486+0000 7f610affd700 10 client.16002 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.486+0000 7f612b7fe700 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7df80e70 0x55ba7df83290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=276b2584b92c5188 server_cookie=6f790078a43986eb in_seq=0 out_seq=0 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f610affd700 1 -- 172.21.15.66:0/3771303662 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba7df3af40 con 0x55ba7df80e70 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f610affd700 10 client.16002 handle_command_reply: tid=0 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 client.16002 shutdown 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 msgr2=0x55ba7df75680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df75680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f612000c5b0 tx=0x7f61200a2480 comp rx=0 tx=0).stop 2023-12-11T19:35:26.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 shutdown_connections 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7df80e70 0x55ba7df83290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 --2- 172.21.15.66:0/3771303662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df75680 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 >> 172.21.15.66:0/3771303662 conn(0x55ba7de9a7e0 msgr2=0x55ba7df66a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.487+0000 7f610affd700 0 client.16002 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.488+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 shutdown_connections 2023-12-11T19:35:26.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.488+0000 7f61459dfb80 1 -- 172.21.15.66:0/3771303662 wait complete. 2023-12-11T19:35:26.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.488+0000 7f61459dfb80 20 client.16002 trim_cache size 0 max 16384 2023-12-11T19:35:26.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.502+0000 7f61459dfb80 1 Processor -- start 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.502+0000 7f61459dfb80 1 -- start start 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6bd30 0x55ba7df7c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 0x55ba7df73e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f61459dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7df6c110 con 0x55ba7df6cc20 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f61459dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7df62b50 con 0x55ba7df6bd30 2023-12-11T19:35:26.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f61459dfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7df7e450 con 0x55ba7df6b3d0 2023-12-11T19:35:26.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f612bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 0x55ba7df73e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f612bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 0x55ba7df73e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51774/0 (socket says 172.21.15.66:51774) 2023-12-11T19:35:26.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f612bfff700 1 -- 172.21.15.66:0/760215060 learned_addr learned my addr 172.21.15.66:0/760215060 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f612b7fe700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.503+0000 7f612affd700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6bd30 0x55ba7df7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612b7fe700 1 -- 172.21.15.66:0/760215060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6bd30 msgr2=0x55ba7df7c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612b7fe700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6bd30 0x55ba7df7c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612b7fe700 1 -- 172.21.15.66:0/760215060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 msgr2=0x55ba7df73e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612b7fe700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 0x55ba7df73e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612b7fe700 1 -- 172.21.15.66:0/760215060 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba7dcd1c60 con 0x55ba7df6b3d0 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612affd700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6bd30 0x55ba7df7c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612bfff700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 0x55ba7df73e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f612b7fe700 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df74650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f611c0099c0 tx=0x7f611c003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6b8be469adb390 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f61127fc700 1 -- 172.21.15.66:0/760215060 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c01d050 con 0x55ba7df6b3d0 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f61127fc700 1 -- 172.21.15.66:0/760215060 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f611c030070 con 0x55ba7df6b3d0 2023-12-11T19:35:26.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.504+0000 7f61127fc700 1 -- 172.21.15.66:0/760215060 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c030ea0 con 0x55ba7df6b3d0 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 -- 172.21.15.66:0/760215060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 msgr2=0x55ba7df74650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df74650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f611c0099c0 tx=0x7f611c003680 comp rx=0 tx=0).stop 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 -- 172.21.15.66:0/760215060 shutdown_connections 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6cc20 0x55ba7df73e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df74650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 --2- 172.21.15.66:0/760215060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6bd30 0x55ba7df7c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 -- 172.21.15.66:0/760215060 >> 172.21.15.66:0/760215060 conn(0x55ba7de9a7e0 msgr2=0x55ba7df73140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 -- 172.21.15.66:0/760215060 shutdown_connections 2023-12-11T19:35:26.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.505+0000 7f61459dfb80 1 -- 172.21.15.66:0/760215060 wait complete. 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.506+0000 7f61459dfb80 1 -- start start 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.506+0000 7f61459dfb80 1 Processor -- start 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df79c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df7a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7df7b700 con 0x55ba7df6bd30 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7df7b870 con 0x55ba7df6cc20 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7df7ba30 con 0x55ba7df6b3d0 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f61459dfb80 10 client.0 _ll_register_callbacks cb 0x55ba7dda4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f612affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df7a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f612affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df7a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51784/0 (socket says 172.21.15.66:51784) 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f612b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df79c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f612affd700 1 -- 172.21.15.66:0/1225761994 learned_addr learned my addr 172.21.15.66:0/1225761994 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.507+0000 7f612bfff700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612affd700 1 -- 172.21.15.66:0/1225761994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 msgr2=0x55ba7df79c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612affd700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df79c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612affd700 1 -- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 msgr2=0x55ba7df7b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612affd700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612affd700 1 -- 172.21.15.66:0/1225761994 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba7dce8de0 con 0x55ba7df6bd30 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612b7fe700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df79c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f612affd700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df7a1b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61200a25a0 tx=0x7f612009e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca77feadee39d58 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f610affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f610affd700 1 -- 172.21.15.66:0/1225761994 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612009a9a0 con 0x55ba7df6bd30 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f610affd700 1 -- 172.21.15.66:0/1225761994 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6120065020 con 0x55ba7df6bd30 2023-12-11T19:35:26.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f611c00bc20 con 0x55ba7df6bd30 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f610affd700 1 -- 172.21.15.66:0/1225761994 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f612009a2a0 con 0x55ba7df6bd30 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f61459dfb80 10 client.16008 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.508+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f61240099e0 con 0x55ba7df6bd30 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f610affd700 1 -- 172.21.15.66:0/1225761994 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f611c00bc20 con 0x55ba7df6bd30 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f610affd700 1 -- 172.21.15.66:0/1225761994 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f612007c020 con 0x55ba7df6bd30 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f61459dfb80 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f61459dfb80 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f61459dfb80 20 client.16008 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f61459dfb80 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7dfd7960 0x55ba7dfd9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f61459dfb80 4 client.16008 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ba7de85fd0 con 0x55ba7dfd7960 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.509+0000 7f612b7fe700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7dfd7960 0x55ba7dfd9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f612b7fe700 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7dfd7960 0x55ba7dfd9d80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f82410842f0951e server_cookie=160bc22563bfa79b in_seq=0 out_seq=0 2023-12-11T19:35:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f610affd700 10 client.16008 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f610affd700 1 -- 172.21.15.66:0/1225761994 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ba7de85fd0 con 0x55ba7dfd7960 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f610affd700 10 client.16008 handle_command_reply: tid=0 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 client.16008 shutdown 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 msgr2=0x55ba7df7a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df7a1b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61200a25a0 tx=0x7f612009e300 comp rx=0 tx=0).stop 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 shutdown_connections 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x55ba7dfd7960 0x55ba7dfd9d80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ba7df6cc20 0x55ba7df7b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba7df6b3d0 0x55ba7df79c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 --2- 172.21.15.66:0/1225761994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba7df6bd30 0x55ba7df7a1b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 >> 172.21.15.66:0/1225761994 conn(0x55ba7de9a7e0 msgr2=0x55ba7df87c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.510+0000 7f610affd700 0 client.16008 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:35:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 shutdown_connections 2023-12-11T19:35:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f61459dfb80 1 -- 172.21.15.66:0/1225761994 wait complete. 2023-12-11T19:35:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f61459dfb80 20 client.16008 trim_cache size 0 max 16384 2023-12-11T19:35:26.517 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61140a1480 msgr2=0x7f61140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f6131fff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61140a1480 0x7f61140a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6124009fd0 tx=0x7f6124009390 comp rx=0 tx=0).stop 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 msgr2=0x7f612c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.511+0000 7f6131fff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 0x7f612c199be0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f612000e9a0 tx=0x7f612000ecb0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 shutdown_connections 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61140a1480 0x7f61140a3940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f612c1037b0 0x7f612c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f612c105340 0x7f612c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 --2- 172.21.15.66:0/64701654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f612c1049a0 0x7f612c199be0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 >> 172.21.15.66:0/64701654 conn(0x7f612c0fed20 msgr2=0x7f612c107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 shutdown_connections 2023-12-11T19:35:26.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.512+0000 7f6131fff700 1 -- 172.21.15.66:0/64701654 wait complete. 2023-12-11T19:35:26.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-11T19:35:26.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.787+0000 7f22e79ff700 1 -- 172.21.15.66:0/3743320601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 msgr2=0x7f22e01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.787+0000 7f22e79ff700 1 --2- 172.21.15.66:0/3743320601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e01043f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009990 tx=0x7f22dc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 -- 172.21.15.66:0/3743320601 shutdown_connections 2023-12-11T19:35:26.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 --2- 172.21.15.66:0/3743320601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 0x7f22e01096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 --2- 172.21.15.66:0/3743320601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e01043f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 --2- 172.21.15.66:0/3743320601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 0x7f22e0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 -- 172.21.15.66:0/3743320601 >> 172.21.15.66:0/3743320601 conn(0x7f22e00fb9f0 msgr2=0x7f22e00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 -- 172.21.15.66:0/3743320601 shutdown_connections 2023-12-11T19:35:26.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.788+0000 7f22e79ff700 1 -- 172.21.15.66:0/3743320601 wait complete. 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 Processor -- start 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 -- start start 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 0x7f22e0199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 0x7f22e01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01a1150 con 0x7f22e0102000 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01a1290 con 0x7f22e0107260 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.789+0000 7f22e79ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01a1420 con 0x7f22e0104930 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e0199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51796/0 (socket says 172.21.15.66:51796) 2023-12-11T19:35:26.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 -- 172.21.15.66:0/2320770702 learned_addr learned my addr 172.21.15.66:0/2320770702 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 0x7f22e01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 0x7f22e0199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 -- 172.21.15.66:0/2320770702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 msgr2=0x7f22e0199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 0x7f22e0199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 -- 172.21.15.66:0/2320770702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 msgr2=0x7f22e01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 0x7f22e01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e579b700 1 -- 172.21.15.66:0/2320770702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0095f0 con 0x7f22e0102000 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 0x7f22e0199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.790+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 0x7f22e01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.791+0000 7f22e579b700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e0199620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f22dc006010 tx=0x7f22dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.791+0000 7f22d67fc700 1 -- 172.21.15.66:0/2320770702 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc01d070 con 0x7f22e0102000 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.791+0000 7f22d67fc700 1 -- 172.21.15.66:0/2320770702 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22dc00bdb0 con 0x7f22e0102000 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.791+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e01a16a0 con 0x7f22e0102000 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.791+0000 7f22d67fc700 1 -- 172.21.15.66:0/2320770702 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00fdf0 con 0x7f22e0102000 2023-12-11T19:35:26.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.791+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01a1ac0 con 0x7f22e0102000 2023-12-11T19:35:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.795+0000 7f22d67fc700 1 -- 172.21.15.66:0/2320770702 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f22dc00b920 con 0x7f22e0102000 2023-12-11T19:35:26.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.796+0000 7f22fb355b80 1 Processor -- start 2023-12-11T19:35:26.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.796+0000 7f22fb355b80 1 -- start start 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22dfd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22dfe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22fb355b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22d57b50 con 0x55ae22dfdc20 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22fb355b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22d55470 con 0x55ae22dfc3d0 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22fb355b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22d50d60 con 0x55ae22dfcd30 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22d67fc700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f22cc0a1480 0x7f22cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22e4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22dfe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22d67fc700 1 -- 172.21.15.66:0/2320770702 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22dc0ce030 con 0x7f22e0102000 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22e4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22dfe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51806/0 (socket says 172.21.15.66:51806) 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22e4f9a700 1 -- 172.21.15.66:0/1444279786 learned_addr learned my addr 172.21.15.66:0/1444279786 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.797+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f22cc0a1480 0x7f22cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e579b700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22dfd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e4f9a700 1 -- 172.21.15.66:0/1444279786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 msgr2=0x55ae22dfd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22dfd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e4f9a700 1 -- 172.21.15.66:0/1444279786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 msgr2=0x55ae22dfc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e4f9a700 1 -- 172.21.15.66:0/1444279786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae22bcd630 con 0x55ae22dfdc20 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e579b700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22dfd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:26.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f22cc0a1480 0x7f22cc0a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009fd0 tx=0x7f22d800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.798+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22dfe0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009de0 tx=0x7f22d00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14b59334e6fe670 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.799+0000 7f22cb7fe700 1 -- 172.21.15.66:0/1444279786 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0011e20 con 0x55ae22dfdc20 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.799+0000 7f22cb7fe700 1 -- 172.21.15.66:0/1444279786 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d00128e0 con 0x55ae22dfdc20 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.799+0000 7f22cb7fe700 1 -- 172.21.15.66:0/1444279786 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000f7b0 con 0x55ae22dfdc20 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.799+0000 7f22fb355b80 1 -- 172.21.15.66:0/1444279786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 msgr2=0x55ae22dfe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.799+0000 7f22fb355b80 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22dfe0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009de0 tx=0x7f22d00116c0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 -- 172.21.15.66:0/1444279786 shutdown_connections 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22dfe0a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 --2- 172.21.15.66:0/1444279786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22dfd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 -- 172.21.15.66:0/1444279786 >> 172.21.15.66:0/1444279786 conn(0x55ae22d2b7e0 msgr2=0x55ae22df8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 -- 172.21.15.66:0/1444279786 shutdown_connections 2023-12-11T19:35:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 -- 172.21.15.66:0/1444279786 wait complete. 2023-12-11T19:35:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.800+0000 7f22fb355b80 1 -- start start 2023-12-11T19:35:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.801+0000 7f22fb355b80 1 Processor -- start 2023-12-11T19:35:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.801+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22e061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 0x55ae22e066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22fb355b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22d0a560 con 0x55ae22dfcd30 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22fb355b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22d0a200 con 0x55ae22dfc3d0 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22fb355b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22bcf920 con 0x55ae22dfdc20 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22fb355b80 10 client.0 _ll_register_callbacks cb 0x55ae22c35a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 0x55ae22e066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 0x55ae22e066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51818/0 (socket says 172.21.15.66:51818) 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 -- 172.21.15.66:0/991770504 learned_addr learned my addr 172.21.15.66:0/991770504 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e579b700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22e061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 -- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 msgr2=0x55ae22e0d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 -- 172.21.15.66:0/991770504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 msgr2=0x55ae22e061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22e061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e4f9a700 1 -- 172.21.15.66:0/991770504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d8008a90 con 0x55ae22dfcd30 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.802+0000 7f22e5f9c700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 0x55ae22e066e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009db0 tx=0x7f22d00126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf5f06e7921d3dd server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22c8ff9700 1 -- 172.21.15.66:0/991770504 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0004240 con 0x55ae22dfcd30 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae22defb40 con 0x55ae22dfcd30 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22c8ff9700 1 -- 172.21.15.66:0/991770504 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22d00043a0 con 0x55ae22dfcd30 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22c8ff9700 1 -- 172.21.15.66:0/991770504 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d0012d60 con 0x55ae22dfcd30 2023-12-11T19:35:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22fb355b80 10 client.16020 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.803+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f22d0009a10 con 0x55ae22dfcd30 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22c8ff9700 1 -- 172.21.15.66:0/991770504 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ae22defb40 con 0x55ae22dfcd30 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22c8ff9700 1 -- 172.21.15.66:0/991770504 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22d000f640 con 0x55ae22dfcd30 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22fb355b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22fb355b80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22fb355b80 20 client.16020 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22fb355b80 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e11e70 0x55ae22e14290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22fb355b80 4 client.16020 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae22dcbf40 con 0x55ae22e11e70 2023-12-11T19:35:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.804+0000 7f22e579b700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e11e70 0x55ae22e14290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.805+0000 7f22c8ff9700 10 client.16020 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.805+0000 7f22e579b700 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e11e70 0x55ae22e14290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7366344aa887e9a server_cookie=9bd53778a7075f5e in_seq=0 out_seq=0 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22c8ff9700 1 -- 172.21.15.66:0/991770504 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae22dcbf40 con 0x55ae22e11e70 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22c8ff9700 10 client.16020 handle_command_reply: tid=0 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 client.16020 shutdown 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 msgr2=0x55ae22e066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 0x55ae22e066e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009db0 tx=0x7f22d00126f0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 shutdown_connections 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e11e70 0x55ae22e14290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22e061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 --2- 172.21.15.66:0/991770504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfcd30 0x55ae22e066e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 >> 172.21.15.66:0/991770504 conn(0x55ae22d2b7e0 msgr2=0x55ae22df7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22c8ff9700 0 client.16020 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 shutdown_connections 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.806+0000 7f22fb355b80 1 -- 172.21.15.66:0/991770504 wait complete. 2023-12-11T19:35:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.807+0000 7f22fb355b80 20 client.16020 trim_cache size 0 max 16384 2023-12-11T19:35:26.811 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:26 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1328876209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.820+0000 7f22fb355b80 1 Processor -- start 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 -- start start 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22e0d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22e061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22dfbf70 con 0x55ae22dfdc20 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22e06e20 con 0x55ae22dfc3d0 2023-12-11T19:35:26.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22fb355b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22e03e20 con 0x55ae22dfcd30 2023-12-11T19:35:26.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22e579b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22e579b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37014/0 (socket says 172.21.15.66:37014) 2023-12-11T19:35:26.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22e5f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22e061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22e579b700 1 -- 172.21.15.66:0/3666269665 learned_addr learned my addr 172.21.15.66:0/3666269665 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.821+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22e0d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22e579b700 1 -- 172.21.15.66:0/3666269665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 msgr2=0x55ae22e0d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22e579b700 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22e0d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22e579b700 1 -- 172.21.15.66:0/3666269665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 msgr2=0x55ae22e061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22e579b700 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22e061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22e579b700 1 -- 172.21.15.66:0/3666269665 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae22b79de0 con 0x55ae22dfc3d0 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22e579b700 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfe000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f22dc097e50 tx=0x7f22dc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a844c46b546641b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22cbfff700 1 -- 172.21.15.66:0/3666269665 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc09ad40 con 0x55ae22dfc3d0 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22cbfff700 1 -- 172.21.15.66:0/3666269665 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22dc0aae90 con 0x55ae22dfc3d0 2023-12-11T19:35:26.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.822+0000 7f22cbfff700 1 -- 172.21.15.66:0/3666269665 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0a36e0 con 0x55ae22dfc3d0 2023-12-11T19:35:26.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 -- 172.21.15.66:0/3666269665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 msgr2=0x55ae22dfe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfe000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f22dc097e50 tx=0x7f22dc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 -- 172.21.15.66:0/3666269665 shutdown_connections 2023-12-11T19:35:26.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfc3d0 0x55ae22dfe000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfdc20 0x55ae22e061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 --2- 172.21.15.66:0/3666269665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfcd30 0x55ae22e0d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 -- 172.21.15.66:0/3666269665 >> 172.21.15.66:0/3666269665 conn(0x55ae22d2b7e0 msgr2=0x55ae22e041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 -- 172.21.15.66:0/3666269665 shutdown_connections 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.823+0000 7f22fb355b80 1 -- 172.21.15.66:0/3666269665 wait complete. 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.824+0000 7f22fb355b80 1 -- start start 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.824+0000 7f22fb355b80 1 Processor -- start 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfc3d0 0x55ae22e0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfcd30 0x55ae22e0b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22e0c960 con 0x55ae22dfc3d0 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22e0cad0 con 0x55ae22dfcd30 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae22e0cc90 con 0x55ae22dfdc20 2023-12-11T19:35:26.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22fb355b80 10 client.0 _ll_register_callbacks cb 0x55ae22c35a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:26.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22e579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfc3d0 0x55ae22e0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22e5f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22e4f9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfcd30 0x55ae22e0b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22e579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfc3d0 0x55ae22e0aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51848/0 (socket says 172.21.15.66:51848) 2023-12-11T19:35:26.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.825+0000 7f22e579b700 1 -- 172.21.15.66:0/305204035 learned_addr learned my addr 172.21.15.66:0/305204035 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:26.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22e4f9a700 1 -- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 msgr2=0x55ae22e0c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22e4f9a700 1 -- 172.21.15.66:0/305204035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfc3d0 msgr2=0x55ae22e0aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfc3d0 0x55ae22e0aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22e4f9a700 1 -- 172.21.15.66:0/305204035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e01a1ac0 con 0x55ae22dfcd30 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22e4f9a700 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfcd30 0x55ae22e0b410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f22d000f9c0 tx=0x7f22d000f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a9ef0122705807 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22c8ff9700 1 -- 172.21.15.66:0/305204035 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0005520 con 0x55ae22dfcd30 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d00128e0 con 0x55ae22dfcd30 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.826+0000 7f22c8ff9700 1 -- 172.21.15.66:0/305204035 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22d0000e40 con 0x55ae22dfcd30 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22c8ff9700 1 -- 172.21.15.66:0/305204035 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d00092d0 con 0x55ae22dfcd30 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 10 client.25757 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ae22b62c60 con 0x55ae22dfcd30 2023-12-11T19:35:26.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22c8ff9700 1 -- 172.21.15.66:0/305204035 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d0046080 con 0x55ae22dfcd30 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22c8ff9700 1 -- 172.21.15.66:0/305204035 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f22d0003b10 con 0x55ae22dfcd30 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 10 client.25757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 20 client.25757 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e68ac0 0x55ae22e6aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 4 client.25757 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.827+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae22d16fd0 con 0x55ae22e68ac0 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.828+0000 7f22e579b700 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e68ac0 0x55ae22e6aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22c8ff9700 10 client.25757 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22e579b700 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e68ac0 0x55ae22e6aee0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52a91944ecd0f855 server_cookie=28c2b5696b1e1a1b in_seq=0 out_seq=0 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22c8ff9700 1 -- 172.21.15.66:0/305204035 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae22d16fd0 con 0x55ae22e68ac0 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22c8ff9700 10 client.25757 handle_command_reply: tid=0 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 client.25757 shutdown 2023-12-11T19:35:26.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfcd30 msgr2=0x55ae22e0b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfcd30 0x55ae22e0b410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f22d000f9c0 tx=0x7f22d000f8b0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 shutdown_connections 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x55ae22e68ac0 0x55ae22e6aee0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae22dfdc20 0x55ae22e0c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ae22dfc3d0 0x55ae22e0aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 --2- 172.21.15.66:0/305204035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ae22dfcd30 0x55ae22e0b410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22c8ff9700 0 client.25757 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 >> 172.21.15.66:0/305204035 conn(0x55ae22d2b7e0 msgr2=0x55ae22e00bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 shutdown_connections 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.829+0000 7f22fb355b80 1 -- 172.21.15.66:0/305204035 wait complete. 2023-12-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22fb355b80 20 client.25757 trim_cache size 0 max 16384 2023-12-11T19:35:26.834 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f22cc0a1480 msgr2=0x7f22cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f22cc0a1480 0x7f22cc0a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009fd0 tx=0x7f22d800b040 comp rx=0 tx=0).stop 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 msgr2=0x7f22e0199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e0199620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f22dc006010 tx=0x7f22dc004730 comp rx=0 tx=0).stop 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 shutdown_connections 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f22cc0a1480 0x7f22cc0a3940 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22e0107260 0x7f22e01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22e0102000 0x7f22e0199620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 --2- 172.21.15.66:0/2320770702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e0104930 0x7f22e0199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:26.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.830+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 >> 172.21.15.66:0/2320770702 conn(0x7f22e00fb9f0 msgr2=0x7f22e01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:26.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.831+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 shutdown_connections 2023-12-11T19:35:26.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:26.831+0000 7f22e79ff700 1 -- 172.21.15.66:0/2320770702 wait complete. 2023-12-11T19:35:26.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-11T19:35:26.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:26 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1328876209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:35:27.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:26 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1328876209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2545894505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 msgr2=0x7f96a4106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2545894505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4106d80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9694009990 tx=0x7f9694009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2545894505 shutdown_connections 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2545894505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a41072c0 0x7f96a41096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2545894505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a4102060 0x7f96a4104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2545894505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4106d80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2545894505 >> 172.21.15.66:0/2545894505 conn(0x7f96a40fb9f0 msgr2=0x7f96a40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2545894505 shutdown_connections 2023-12-11T19:35:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.081+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2545894505 wait complete. 2023-12-11T19:35:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.082+0000 7f96ac1ff700 1 Processor -- start 2023-12-11T19:35:27.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.082+0000 7f96ac1ff700 1 -- start start 2023-12-11T19:35:27.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96ac1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a4102060 0x7f96a41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96ac1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96ac1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a41072c0 0x7f96a41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96ac1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41a11f0 con 0x7f96a4104990 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96ac1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41a1330 con 0x7f96a4102060 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96ac1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41a14c0 con 0x7f96a41072c0 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96a979a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96a979a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51870/0 (socket says 172.21.15.66:51870) 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96a979a700 1 -- 172.21.15.66:0/2462337228 learned_addr learned my addr 172.21.15.66:0/2462337228 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a4102060 0x7f96a41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.083+0000 7f96aa79c700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a41072c0 0x7f96a41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a979a700 1 -- 172.21.15.66:0/2462337228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a41072c0 msgr2=0x7f96a41a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a979a700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a41072c0 0x7f96a41a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a979a700 1 -- 172.21.15.66:0/2462337228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a4102060 msgr2=0x7f96a41996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a979a700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a4102060 0x7f96a41996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a979a700 1 -- 172.21.15.66:0/2462337228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940095f0 con 0x7f96a4104990 2023-12-11T19:35:27.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a4102060 0x7f96a41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:27.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96a979a700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4199c00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f969400b2e0 tx=0x7f9694004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f969affd700 1 -- 172.21.15.66:0/2462337228 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969401d070 con 0x7f96a4104990 2023-12-11T19:35:27.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f969affd700 1 -- 172.21.15.66:0/2462337228 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969400beb0 con 0x7f96a4104990 2023-12-11T19:35:27.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a41a1740 con 0x7f96a4104990 2023-12-11T19:35:27.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f969affd700 1 -- 172.21.15.66:0/2462337228 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694021410 con 0x7f96a4104990 2023-12-11T19:35:27.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.084+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a41a1b60 con 0x7f96a4104990 2023-12-11T19:35:27.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.089+0000 7f969affd700 1 -- 172.21.15.66:0/2462337228 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f969400f640 con 0x7f96a4104990 2023-12-11T19:35:27.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.090+0000 7f96bfbbcb80 1 Processor -- start 2023-12-11T19:35:27.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.090+0000 7f96bfbbcb80 1 -- start start 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e5433d0 0x55fa3e5437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e543d30 0x55fa3e544640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e5450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e49eb50 con 0x55fa3e543d30 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e49c470 con 0x55fa3e544c20 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e497d60 con 0x55fa3e5433d0 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f969affd700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f96900a1480 0x7f96900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96a9f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e543d30 0x55fa3e544640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96a9f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e543d30 0x55fa3e544640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51872/0 (socket says 172.21.15.66:51872) 2023-12-11T19:35:27.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96a9f9b700 1 -- 172.21.15.66:0/1739865823 learned_addr learned my addr 172.21.15.66:0/1739865823 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96aa79c700 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e5433d0 0x55fa3e5437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f969affd700 1 -- 172.21.15.66:0/2462337228 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96940ce0c0 con 0x7f96a4104990 2023-12-11T19:35:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.091+0000 7f96a979a700 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e5450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96aa79c700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f96900a1480 0x7f96900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96a979a700 1 -- 172.21.15.66:0/1739865823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e5433d0 msgr2=0x55fa3e5437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96a979a700 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e5433d0 0x55fa3e5437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96a979a700 1 -- 172.21.15.66:0/1739865823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e543d30 msgr2=0x55fa3e544640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96a979a700 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e543d30 0x55fa3e544640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96a979a700 1 -- 172.21.15.66:0/1739865823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa3e314630 con 0x55fa3e544c20 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f96a979a700 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e5450a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9694006010 tx=0x7f9694009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a6440b60e198bfc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.092+0000 7f968ffff700 1 -- 172.21.15.66:0/1739865823 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969409add0 con 0x55fa3e544c20 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.093+0000 7f968ffff700 1 -- 172.21.15.66:0/1739865823 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969400f460 con 0x55fa3e544c20 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.093+0000 7f96aa79c700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f96900a1480 0x7f96900a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f969c009690 tx=0x7f969c009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.093+0000 7f968ffff700 1 -- 172.21.15.66:0/1739865823 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96940d0110 con 0x55fa3e544c20 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/1739865823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 msgr2=0x55fa3e5450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e5450a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9694006010 tx=0x7f9694009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/1739865823 shutdown_connections 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e5450a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e5433d0 0x55fa3e5437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/1739865823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e543d30 0x55fa3e544640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/1739865823 >> 172.21.15.66:0/1739865823 conn(0x55fa3e4727e0 msgr2=0x55fa3e53fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/1739865823 shutdown_connections 2023-12-11T19:35:27.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.094+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/1739865823 wait complete. 2023-12-11T19:35:27.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.095+0000 7f96bfbbcb80 1 -- start start 2023-12-11T19:35:27.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.095+0000 7f96bfbbcb80 1 Processor -- start 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e555630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e543d30 0x55fa3e555b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e544c20 0x55fa3e55cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e451560 con 0x55fa3e5433d0 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e451200 con 0x55fa3e543d30 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e316920 con 0x55fa3e544c20 2023-12-11T19:35:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96bfbbcb80 10 client.0 _ll_register_callbacks cb 0x55fa3e37ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96a9f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e555630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96a9f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e555630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51876/0 (socket says 172.21.15.66:51876) 2023-12-11T19:35:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.096+0000 7f96a9f9b700 1 -- 172.21.15.66:0/3846050373 learned_addr learned my addr 172.21.15.66:0/3846050373 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96a979a700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e543d30 0x55fa3e555b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96aa79c700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e544c20 0x55fa3e55cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96a9f9b700 1 -- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e544c20 msgr2=0x55fa3e55cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e544c20 0x55fa3e55cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96a9f9b700 1 -- 172.21.15.66:0/3846050373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e543d30 msgr2=0x55fa3e555b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e543d30 0x55fa3e555b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.097+0000 7f96a9f9b700 1 -- 172.21.15.66:0/3846050373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a41a1b60 con 0x55fa3e5433d0 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.098+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e555630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f96a0012d20 tx=0x7f96a000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe14d8801cda550 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.098+0000 7f968d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.098+0000 7f968d7fa700 1 -- 172.21.15.66:0/3846050373 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x55fa3e5433d0 2023-12-11T19:35:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.099+0000 7f968d7fa700 1 -- 172.21.15.66:0/3846050373 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96a000fc30 con 0x55fa3e5433d0 2023-12-11T19:35:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.099+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa3e536b40 con 0x55fa3e5433d0 2023-12-11T19:35:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.099+0000 7f968d7fa700 1 -- 172.21.15.66:0/3846050373 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a000fdf0 con 0x55fa3e5433d0 2023-12-11T19:35:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.099+0000 7f96bfbbcb80 10 client.16038 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.099+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fa3e2c0de0 con 0x55fa3e5433d0 2023-12-11T19:35:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.099+0000 7f968d7fa700 1 -- 172.21.15.66:0/3846050373 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fa3e536b40 con 0x55fa3e5433d0 2023-12-11T19:35:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f968d7fa700 1 -- 172.21.15.66:0/3846050373 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f96a0003b40 con 0x55fa3e5433d0 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f96bfbbcb80 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f96bfbbcb80 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f96bfbbcb80 20 client.16038 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e561480 0x55fa3e5638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f96bfbbcb80 4 client.16038 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.100+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa3e512f40 con 0x55fa3e561480 2023-12-11T19:35:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.101+0000 7f96a979a700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e561480 0x55fa3e5638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.102+0000 7f968d7fa700 10 client.16038 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:35:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.102+0000 7f96a979a700 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e561480 0x55fa3e5638a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ebe2dec947aa33b server_cookie=a492d68db63f2367 in_seq=0 out_seq=0 2023-12-11T19:35:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.103+0000 7f968d7fa700 1 -- 172.21.15.66:0/3846050373 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa3e512f40 con 0x55fa3e561480 2023-12-11T19:35:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.103+0000 7f968d7fa700 10 client.16038 handle_command_reply: tid=0 2023-12-11T19:35:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.103+0000 7f96bfbbcb80 1 client.16038 shutdown 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.103+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 msgr2=0x55fa3e555630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.103+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e555630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f96a0012d20 tx=0x7f96a000f6a0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 shutdown_connections 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e544c20 0x55fa3e55cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e555630 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e561480 0x55fa3e5638a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/3846050373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e543d30 0x55fa3e555b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 >> 172.21.15.66:0/3846050373 conn(0x55fa3e4727e0 msgr2=0x55fa3e53ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f968d7fa700 0 client.16038 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 shutdown_connections 2023-12-11T19:35:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/3846050373 wait complete. 2023-12-11T19:35:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.104+0000 7f96bfbbcb80 20 client.16038 trim_cache size 0 max 16384 2023-12-11T19:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.120+0000 7f96bfbbcb80 1 Processor -- start 2023-12-11T19:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.120+0000 7f96bfbbcb80 1 -- start start 2023-12-11T19:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.120+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e545000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.120+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e568240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e536b40 con 0x55fa3e5433d0 2023-12-11T19:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e55dd30 con 0x55fa3e544c20 2023-12-11T19:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e54be00 con 0x55fa3e543d30 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e545000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e545000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51880/0 (socket says 172.21.15.66:51880) 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2676991784 learned_addr learned my addr 172.21.15.66:0/2676991784 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96aa79c700 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e568240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a979a700 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2676991784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 msgr2=0x55fa3e55cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2676991784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 msgr2=0x55fa3e568240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e568240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.121+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2676991784 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa3e2a9c60 con 0x55fa3e5433d0 2023-12-11T19:35:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.122+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e545000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f96a0003e70 tx=0x7f96a0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48ac60173736b70 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.122+0000 7f9698ff9700 1 -- 172.21.15.66:0/2676991784 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017030 con 0x55fa3e5433d0 2023-12-11T19:35:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.122+0000 7f9698ff9700 1 -- 172.21.15.66:0/2676991784 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96a0018040 con 0x55fa3e5433d0 2023-12-11T19:35:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.122+0000 7f9698ff9700 1 -- 172.21.15.66:0/2676991784 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0003680 con 0x55fa3e5433d0 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2676991784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 msgr2=0x55fa3e545000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e545000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f96a0003e70 tx=0x7f96a0009f80 comp rx=0 tx=0).stop 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2676991784 shutdown_connections 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e544c20 0x55fa3e568240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e5433d0 0x55fa3e545000 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2676991784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2676991784 >> 172.21.15.66:0/2676991784 conn(0x55fa3e4727e0 msgr2=0x55fa3e54b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2676991784 shutdown_connections 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2676991784 wait complete. 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.123+0000 7f96bfbbcb80 1 -- start start 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 Processor -- start 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e5433d0 0x55fa3e55a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 0x55fa3e55bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e55c150 con 0x55fa3e544c20 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e55c2c0 con 0x55fa3e5433d0 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa3e55c480 con 0x55fa3e543d30 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96bfbbcb80 10 client.0 _ll_register_callbacks cb 0x55fa3e37ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.124+0000 7f96aa79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 0x55fa3e55bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96aa79c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 0x55fa3e55bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51882/0 (socket says 172.21.15.66:51882) 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96aa79c700 1 -- 172.21.15.66:0/2948691854 learned_addr learned my addr 172.21.15.66:0/2948691854 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e5433d0 0x55fa3e55a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a979a700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 msgr2=0x55fa3e55acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2948691854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 msgr2=0x55fa3e55bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 0x55fa3e55bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 -- 172.21.15.66:0/2948691854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa3e2c0de0 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96aa79c700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 0x55fa3e55bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f96a9f9b700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e5433d0 0x55fa3e55a7b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96a0005a80 tx=0x7f96a0010b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c0c2d90d7014f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.125+0000 7f968d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.126+0000 7f968d7fa700 1 -- 172.21.15.66:0/2948691854 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017040 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.126+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f969400f460 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.126+0000 7f968d7fa700 1 -- 172.21.15.66:0/2948691854 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96a002b070 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.126+0000 7f968d7fa700 1 -- 172.21.15.66:0/2948691854 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a0009260 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.126+0000 7f96bfbbcb80 10 client.25775 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.126+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fa3e2e7250 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f968d7fa700 1 -- 172.21.15.66:0/2948691854 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f969400f460 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f968d7fa700 1 -- 172.21.15.66:0/2948691854 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f96a0039070 con 0x55fa3e5433d0 2023-12-11T19:35:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96bfbbcb80 10 client.25775 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96bfbbcb80 10 client.25775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:35:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96bfbbcb80 20 client.25775 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e5b8370 0x55fa3e5ba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96bfbbcb80 4 client.25775 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:35:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa3e45dfd0 con 0x55fa3e5b8370 2023-12-11T19:35:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.127+0000 7f96a979a700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e5b8370 0x55fa3e5ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f96a979a700 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e5b8370 0x55fa3e5ba790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a14f23091fd06e41 server_cookie=597a66857fd95578 in_seq=0 out_seq=0 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f968d7fa700 10 client.25775 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f968d7fa700 1 -- 172.21.15.66:0/2948691854 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa3e45dfd0 con 0x55fa3e5b8370 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f968d7fa700 10 client.25775 handle_command_reply: tid=0 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f96bfbbcb80 1 client.25775 shutdown 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e5433d0 msgr2=0x55fa3e55a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.128+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e5433d0 0x55fa3e55a7b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96a0005a80 tx=0x7f96a0010b60 comp rx=0 tx=0).stop 2023-12-11T19:35:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 shutdown_connections 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x55fa3e5b8370 0x55fa3e5ba790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fa3e5433d0 0x55fa3e55a7b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fa3e544c20 0x55fa3e55bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f968d7fa700 0 client.25775 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 --2- 172.21.15.66:0/2948691854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fa3e543d30 0x55fa3e55acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 >> 172.21.15.66:0/2948691854 conn(0x55fa3e4727e0 msgr2=0x55fa3e547b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 shutdown_connections 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 1 -- 172.21.15.66:0/2948691854 wait complete. 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96bfbbcb80 20 client.25775 trim_cache size 0 max 16384 2023-12-11T19:35:27.132 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:35:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f96900a1480 msgr2=0x7f96900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f96900a1480 0x7f96900a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f969c009690 tx=0x7f969c009510 comp rx=0 tx=0).stop 2023-12-11T19:35:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 msgr2=0x7f96a4199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.129+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4199c00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f969400b2e0 tx=0x7f9694004730 comp rx=0 tx=0).stop 2023-12-11T19:35:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 shutdown_connections 2023-12-11T19:35:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f96900a1480 0x7f96900a3940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96a4102060 0x7f96a41996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a41072c0 0x7f96a41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 --2- 172.21.15.66:0/2462337228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96a4104990 0x7f96a4199c00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 >> 172.21.15.66:0/2462337228 conn(0x7f96a40fb9f0 msgr2=0x7f96a41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 shutdown_connections 2023-12-11T19:35:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.130+0000 7f96ac1ff700 1 -- 172.21.15.66:0/2462337228 wait complete. 2023-12-11T19:35:27.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-11T19:35:27.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 -- 172.21.15.66:0/2709270447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0102790 msgr2=0x7ff9c0102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/2709270447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0102790 0x7ff9c0102bf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00099c0 tx=0x7ff9b0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 -- 172.21.15.66:0/2709270447 shutdown_connections 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/2709270447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c01015a0 0x7ff9c01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/2709270447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c0103130 0x7ff9c01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/2709270447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0102790 0x7ff9c0102bf0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 -- 172.21.15.66:0/2709270447 >> 172.21.15.66:0/2709270447 conn(0x7ff9c00fcb10 msgr2=0x7ff9c00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 -- 172.21.15.66:0/2709270447 shutdown_connections 2023-12-11T19:35:27.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.412+0000 7ff9c85ff700 1 -- 172.21.15.66:0/2709270447 wait complete. 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.413+0000 7ff9c85ff700 1 Processor -- start 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 -- start start 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c01015a0 0x7ff9c0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c0102790 0x7ff9c00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 0x7ff9c0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c019b310 con 0x7ff9c0103130 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c0074440 con 0x7ff9c0102790 2023-12-11T19:35:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c85ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c0074600 con 0x7ff9c01015a0 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c6b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 0x7ff9c0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c5b9a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c0102790 0x7ff9c00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c6b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 0x7ff9c0073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51902/0 (socket says 172.21.15.66:51902) 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c6b9c700 1 -- 172.21.15.66:0/3133424033 learned_addr learned my addr 172.21.15.66:0/3133424033 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.414+0000 7ff9c639b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c01015a0 0x7ff9c0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/3133424033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c01015a0 msgr2=0x7ff9c0075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c01015a0 0x7ff9c0075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 msgr2=0x7ff9c0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 0x7ff9c0073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/3133424033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00095f0 con 0x7ff9c0102790 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c6b9c700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 0x7ff9c0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.415+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c0102790 0x7ff9c00739c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009f50 tx=0x7ff9b0003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.416+0000 7ff9b77fe700 1 -- 172.21.15.66:0/3133424033 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b001d070 con 0x7ff9c0102790 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.416+0000 7ff9b77fe700 1 -- 172.21.15.66:0/3133424033 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9b000bda0 con 0x7ff9c0102790 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.416+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c0074820 con 0x7ff9c0102790 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.416+0000 7ff9b77fe700 1 -- 172.21.15.66:0/3133424033 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0022bb0 con 0x7ff9c0102790 2023-12-11T19:35:27.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.416+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c01a5fb0 con 0x7ff9c0102790 2023-12-11T19:35:27.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.420+0000 7ff9b77fe700 1 -- 172.21.15.66:0/3133424033 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff9b0003e10 con 0x7ff9c0102790 2023-12-11T19:35:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.420+0000 7ff9dbf71b80 1 Processor -- start 2023-12-11T19:35:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.420+0000 7ff9dbf71b80 1 -- start start 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc2817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc282640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc2830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc1dcb50 con 0x5637cc2813d0 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc1da470 con 0x5637cc281d30 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc1d5d60 con 0x5637cc282c20 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9c6b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc2817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9c6b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc2817f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51910/0 (socket says 172.21.15.66:51910) 2023-12-11T19:35:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9c6b9c700 1 -- 172.21.15.66:0/3719847088 learned_addr learned my addr 172.21.15.66:0/3719847088 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc282640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.421+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc2830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c639b700 1 -- 172.21.15.66:0/3719847088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 msgr2=0x5637cc2830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc2830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c639b700 1 -- 172.21.15.66:0/3719847088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 msgr2=0x5637cc2817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc2817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c639b700 1 -- 172.21.15.66:0/3719847088 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637cc052630 con 0x5637cc281d30 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9b77fe700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9ac0a1500 0x7ff9ac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc282640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc007c30 tx=0x7ff9bc007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc9582efa543bd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9b77fe700 1 -- 172.21.15.66:0/3133424033 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9b00cdfc0 con 0x7ff9c0102790 2023-12-11T19:35:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9c6b9c700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9ac0a1500 0x7ff9ac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9b4bfe700 1 -- 172.21.15.66:0/3719847088 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc010ca0 con 0x5637cc281d30 2023-12-11T19:35:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9b4bfe700 1 -- 172.21.15.66:0/3719847088 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9bc0044e0 con 0x5637cc281d30 2023-12-11T19:35:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.422+0000 7ff9b4bfe700 1 -- 172.21.15.66:0/3719847088 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc005280 con 0x5637cc281d30 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3719847088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 msgr2=0x5637cc282640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9c6b9c700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9ac0a1500 0x7ff9ac0a39c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8007990 tx=0x7ff9b8009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc282640 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc007c30 tx=0x7ff9bc007f40 comp rx=0 tx=0).stop 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3719847088 shutdown_connections 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc2830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc2817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3719847088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc282640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3719847088 >> 172.21.15.66:0/3719847088 conn(0x5637cc1b07e0 msgr2=0x5637cc27dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3719847088 shutdown_connections 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.423+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3719847088 wait complete. 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.424+0000 7ff9dbf71b80 1 -- start start 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.424+0000 7ff9dbf71b80 1 Processor -- start 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.424+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc2813d0 0x5637cc28b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc28b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc282c20 0x5637cc292780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc18f560 con 0x5637cc281d30 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc18f200 con 0x5637cc282c20 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc054920 con 0x5637cc2813d0 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9dbf71b80 10 client.0 _ll_register_callbacks cb 0x5637cc0baa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc28b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc28b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51918/0 (socket says 172.21.15.66:51918) 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/285266528 learned_addr learned my addr 172.21.15.66:0/285266528 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c639b700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc2813d0 0x5637cc28b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c6b9c700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc282c20 0x5637cc292780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/285266528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc2813d0 msgr2=0x5637cc28b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc2813d0 0x5637cc28b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc282c20 msgr2=0x5637cc292780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc282c20 0x5637cc292780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.425+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/285266528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc007890 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc28b740 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff9b009b4c0 tx=0x7ff9b009ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5090271511732b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/285266528 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b009b300 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/285266528 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9b009bc30 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637cc274b40 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/285266528 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b009b070 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9dbf71b80 10 client.16056 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5637cbffede0 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.426+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/285266528 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637cc274b40 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/285266528 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff9b00d00d0 con 0x5637cc281d30 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9dbf71b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9dbf71b80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9dbf71b80 20 client.16056 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc296ba0 0x5637cc298fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9dbf71b80 4 client.16056 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637cc250f40 con 0x5637cc296ba0 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.427+0000 7ff9c639b700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc296ba0 0x5637cc298fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.428+0000 7ff9a9ffb700 10 client.16056 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.428+0000 7ff9c639b700 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc296ba0 0x5637cc298fc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a13472986fd7cf4 server_cookie=eb40293855a79747 in_seq=0 out_seq=0 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/285266528 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637cc250f40 con 0x5637cc296ba0 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9a9ffb700 10 client.16056 handle_command_reply: tid=0 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 client.16056 shutdown 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 msgr2=0x5637cc28b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc28b740 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff9b009b4c0 tx=0x7ff9b009ba70 comp rx=0 tx=0).stop 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 shutdown_connections 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc296ba0 0x5637cc298fc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc282c20 0x5637cc292780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc2813d0 0x5637cc28b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/285266528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc28b740 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 >> 172.21.15.66:0/285266528 conn(0x5637cc1b07e0 msgr2=0x5637cc27ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9a9ffb700 0 client.16056 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 shutdown_connections 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.429+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/285266528 wait complete. 2023-12-11T19:35:27.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.430+0000 7ff9dbf71b80 20 client.16056 trim_cache size 0 max 16384 2023-12-11T19:35:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.443+0000 7ff9dbf71b80 1 Processor -- start 2023-12-11T19:35:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.443+0000 7ff9dbf71b80 1 -- start start 2023-12-11T19:35:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc2813d0 0x5637cc2891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc292740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc29d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc293900 con 0x5637cc281d30 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc27e200 con 0x5637cc2813d0 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc282110 con 0x5637cc282c20 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c5b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc292740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c5b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc292740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51922/0 (socket says 172.21.15.66:51922) 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c5b9a700 1 -- 172.21.15.66:0/2673284819 learned_addr learned my addr 172.21.15.66:0/2673284819 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c639b700 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc2813d0 0x5637cc2891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c6b9c700 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc29d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c639b700 1 -- 172.21.15.66:0/2673284819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 msgr2=0x5637cc29d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c639b700 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc29d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c639b700 1 -- 172.21.15.66:0/2673284819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 msgr2=0x5637cc292740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c639b700 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc292740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.444+0000 7ff9c639b700 1 -- 172.21.15.66:0/2673284819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637cbfe7c60 con 0x5637cc2813d0 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.445+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc292740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.445+0000 7ff9c639b700 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc2813d0 0x5637cc2891d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc007fd0 tx=0x7ff9bc005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3df42921d9ca97c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.445+0000 7ff9b53ff700 1 -- 172.21.15.66:0/2673284819 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc01d040 con 0x5637cc2813d0 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.445+0000 7ff9b53ff700 1 -- 172.21.15.66:0/2673284819 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9bc01e040 con 0x5637cc2813d0 2023-12-11T19:35:27.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.445+0000 7ff9b53ff700 1 -- 172.21.15.66:0/2673284819 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0042c0 con 0x5637cc2813d0 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/2673284819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc2813d0 msgr2=0x5637cc2891d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc2813d0 0x5637cc2891d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc007fd0 tx=0x7ff9bc005d80 comp rx=0 tx=0).stop 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/2673284819 shutdown_connections 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc2813d0 0x5637cc2891d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc29d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/2673284819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc281d30 0x5637cc292740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/2673284819 >> 172.21.15.66:0/2673284819 conn(0x5637cc1b07e0 msgr2=0x5637cc28a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/2673284819 shutdown_connections 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/2673284819 wait complete. 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.446+0000 7ff9dbf71b80 1 -- start start 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 Processor -- start 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc290030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc290570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc291580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc291ac0 con 0x5637cc2813d0 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc291c30 con 0x5637cc281d30 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.447+0000 7ff9dbf71b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cc291df0 con 0x5637cc282c20 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9dbf71b80 10 client.0 _ll_register_callbacks cb 0x5637cc0baa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc290030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc290030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51936/0 (socket says 172.21.15.66:51936) 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 -- 172.21.15.66:0/3581847119 learned_addr learned my addr 172.21.15.66:0/3581847119 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:27.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc290570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c6b9c700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc291580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 -- 172.21.15.66:0/3581847119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 msgr2=0x5637cc291580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc291580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 -- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 msgr2=0x5637cc290570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc290570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.448+0000 7ff9c639b700 1 -- 172.21.15.66:0/3581847119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637cbffede0 con 0x5637cc2813d0 2023-12-11T19:35:27.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9c639b700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc290030 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc01b040 tx=0x7ff9bc007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c274ea8f09d6524a server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/3581847119 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc010aa0 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9bc0044e0 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/3581847119 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9bc010c00 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/3581847119 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9bc010dc0 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9dbf71b80 10 client.16062 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.449+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5637cc052630 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/3581847119 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9bc0044e0 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/3581847119 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff9bc0103d0 con 0x5637cc2813d0 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9dbf71b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9dbf71b80 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9dbf71b80 20 client.16062 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc2ed850 0x5637cc2efc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:27.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9dbf71b80 4 client.16062 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.450+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637cc19bfd0 con 0x5637cc2ed850 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc2ed850 0x5637cc2efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9c5b9a700 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc2ed850 0x5637cc2efc70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5196271970495 server_cookie=a24a69437d62abac in_seq=0 out_seq=0 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9a9ffb700 10 client.16062 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9a9ffb700 1 -- 172.21.15.66:0/3581847119 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637cc19bfd0 con 0x5637cc2ed850 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9a9ffb700 10 client.16062 handle_command_reply: tid=0 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9dbf71b80 1 client.16062 shutdown 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 msgr2=0x5637cc290030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.451+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc290030 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc01b040 tx=0x7ff9bc007490 comp rx=0 tx=0).stop 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 shutdown_connections 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5637cc2ed850 0x5637cc2efc70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637cc282c20 0x5637cc291580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637cc2813d0 0x5637cc290030 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 --2- 172.21.15.66:0/3581847119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5637cc281d30 0x5637cc290570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 >> 172.21.15.66:0/3581847119 conn(0x5637cc1b07e0 msgr2=0x5637cc289190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9a9ffb700 0 client.16062 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:27.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 shutdown_connections 2023-12-11T19:35:27.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 1 -- 172.21.15.66:0/3581847119 wait complete. 2023-12-11T19:35:27.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.452+0000 7ff9dbf71b80 20 client.16062 trim_cache size 0 max 16384 2023-12-11T19:35:27.458 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9ac0a1500 msgr2=0x7ff9ac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9ac0a1500 0x7ff9ac0a39c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8007990 tx=0x7ff9b8009040 comp rx=0 tx=0).stop 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c0102790 msgr2=0x7ff9c00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c0102790 0x7ff9c00739c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009f50 tx=0x7ff9b0003a50 comp rx=0 tx=0).stop 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 shutdown_connections 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9ac0a1500 0x7ff9ac0a39c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9c0103130 0x7ff9c0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c01015a0 0x7ff9c0075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 --2- 172.21.15.66:0/3133424033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c0102790 0x7ff9c00739c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 >> 172.21.15.66:0/3133424033 conn(0x7ff9c00fcb10 msgr2=0x7ff9c0105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:27.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 shutdown_connections 2023-12-11T19:35:27.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:27.453+0000 7ff9c85ff700 1 -- 172.21.15.66:0/3133424033 wait complete. 2023-12-11T19:35:27.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:27 smithi066 ceph-mon[97920]: pgmap v416: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 37 MiB/s wr, 23 op/s 2023-12-11T19:35:27.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:27 smithi070 ceph-mon[104044]: pgmap v416: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 37 MiB/s wr, 23 op/s 2023-12-11T19:35:28.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:27 smithi167 ceph-mon[102750]: pgmap v416: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 37 MiB/s wr, 23 op/s 2023-12-11T19:35:28.475 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-11T19:35:28.476 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-11T19:35:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 -- 172.21.15.66:0/1143066181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b940ffdf0 msgr2=0x7f3b94102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 --2- 172.21.15.66:0/1143066181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b940ffdf0 0x7f3b94102220 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009990 tx=0x7f3b84009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 -- 172.21.15.66:0/1143066181 shutdown_connections 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 --2- 172.21.15.66:0/1143066181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b94105130 0x7f3b941055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 --2- 172.21.15.66:0/1143066181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b940ffdf0 0x7f3b94102220 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 --2- 172.21.15.66:0/1143066181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b94102760 0x7f3b94104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 -- 172.21.15.66:0/1143066181 >> 172.21.15.66:0/1143066181 conn(0x7f3b940fb9f0 msgr2=0x7f3b940fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 -- 172.21.15.66:0/1143066181 shutdown_connections 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.719+0000 7f3b98dff700 1 -- 172.21.15.66:0/1143066181 wait complete. 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 Processor -- start 2023-12-11T19:35:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 -- start start 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b940ffdf0 0x7f3b9419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b94102760 0x7f3b9419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 0x7f3b941a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a5560 con 0x7f3b94105130 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a56a0 con 0x7f3b940ffdf0 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.721+0000 7f3b98dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941a5830 con 0x7f3b94102760 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 0x7f3b941a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 0x7f3b941a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51952/0 (socket says 172.21.15.66:51952) 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 -- 172.21.15.66:0/502608183 learned_addr learned my addr 172.21.15.66:0/502608183 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b9259c700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b940ffdf0 0x7f3b9419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b94102760 0x7f3b9419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 -- 172.21.15.66:0/502608183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b94102760 msgr2=0x7f3b9419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b94102760 0x7f3b9419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 -- 172.21.15.66:0/502608183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b940ffdf0 msgr2=0x7f3b9419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b940ffdf0 0x7f3b9419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b92d9d700 1 -- 172.21.15.66:0/502608183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840095f0 con 0x7f3b94105130 2023-12-11T19:35:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.722+0000 7f3b9259c700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b940ffdf0 0x7f3b9419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.723+0000 7f3b92d9d700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 0x7f3b941a4f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ef30 tx=0x7f3b8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.723+0000 7f3b837fe700 1 -- 172.21.15.66:0/502608183 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c018070 con 0x7f3b94105130 2023-12-11T19:35:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.723+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b941a5ab0 con 0x7f3b94105130 2023-12-11T19:35:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.723+0000 7f3b837fe700 1 -- 172.21.15.66:0/502608183 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b8c00f040 con 0x7f3b94105130 2023-12-11T19:35:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.723+0000 7f3b837fe700 1 -- 172.21.15.66:0/502608183 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00cbe0 con 0x7f3b94105130 2023-12-11T19:35:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.723+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b941a5fc0 con 0x7f3b94105130 2023-12-11T19:35:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.727+0000 7f3b837fe700 1 -- 172.21.15.66:0/502608183 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3b8c00cd40 con 0x7f3b94105130 2023-12-11T19:35:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.728+0000 7f3bac692b80 1 Processor -- start 2023-12-11T19:35:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.728+0000 7f3bac692b80 1 -- start start 2023-12-11T19:35:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.728+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f924969870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92496a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92496b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3bac692b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9248c4bd0 con 0x55f92496aca0 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3bac692b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9248c24f0 con 0x55f924969450 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3bac692b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9248bdde0 con 0x55f924969db0 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f924969870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f924969870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37182/0 (socket says 172.21.15.66:37182) 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 -- 172.21.15.66:0/1569533745 learned_addr learned my addr 172.21.15.66:0/1569533745 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b9259c700 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92496a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92496b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b837fe700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b7c0a14d0 0x7f3b7c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 -- 172.21.15.66:0/1569533745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 msgr2=0x55f92496a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92496a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 -- 172.21.15.66:0/1569533745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 msgr2=0x55f92496b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92496b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.729+0000 7f3b91d9b700 1 -- 172.21.15.66:0/1569533745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f92473a6b0 con 0x55f924969450 2023-12-11T19:35:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.730+0000 7f3b837fe700 1 -- 172.21.15.66:0/502608183 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b8c0cc7a0 con 0x7f3b94105130 2023-12-11T19:35:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.730+0000 7f3b92d9d700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b7c0a14d0 0x7f3b7c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.730+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f924969870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3b880118b0 tx=0x7f3b88009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3451080a168a29da server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.730+0000 7f3b80ff9700 1 -- 172.21.15.66:0/1569533745 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004030 con 0x55f924969450 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.730+0000 7f3b80ff9700 1 -- 172.21.15.66:0/1569533745 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b88004830 con 0x55f924969450 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.730+0000 7f3b80ff9700 1 -- 172.21.15.66:0/1569533745 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88005280 con 0x55f924969450 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 -- 172.21.15.66:0/1569533745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 msgr2=0x55f924969870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3b92d9d700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b7c0a14d0 0x7f3b7c0a3990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009c10 tx=0x7f3b8c004bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f924969870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3b880118b0 tx=0x7f3b88009880 comp rx=0 tx=0).stop 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 -- 172.21.15.66:0/1569533745 shutdown_connections 2023-12-11T19:35:28.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f924969870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92496b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1569533745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92496a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 -- 172.21.15.66:0/1569533745 >> 172.21.15.66:0/1569533745 conn(0x55f924898860 msgr2=0x55f924965e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 -- 172.21.15.66:0/1569533745 shutdown_connections 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.731+0000 7f3bac692b80 1 -- 172.21.15.66:0/1569533745 wait complete. 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.732+0000 7f3bac692b80 1 -- start start 2023-12-11T19:35:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.732+0000 7f3bac692b80 1 Processor -- start 2023-12-11T19:35:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9248775e0 con 0x55f92496aca0 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f924877280 con 0x55f924969db0 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92473c9a0 con 0x55f924969450 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3bac692b80 10 client.0 _ll_register_callbacks cb 0x55f9247a2b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3b9259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3b91d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3b9259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37184/0 (socket says 172.21.15.66:37184) 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.733+0000 7f3b9259c700 1 -- 172.21.15.66:0/1330270355 learned_addr learned my addr 172.21.15.66:0/1330270355 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b9259c700 1 -- 172.21.15.66:0/1330270355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969450 msgr2=0x55f9249732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b9259c700 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b9259c700 1 -- 172.21.15.66:0/1330270355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 msgr2=0x55f92497a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b9259c700 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b9259c700 1 -- 172.21.15.66:0/1330270355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b88019040 con 0x55f924969db0 2023-12-11T19:35:28.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b9259c700 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 0x55f924973830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b84003f60 tx=0x7f3b84003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d64fa48cb456c8f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:28.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.734+0000 7f3b79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:28.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3b79ffb700 1 -- 172.21.15.66:0/1330270355 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401d070 con 0x55f924969db0 2023-12-11T19:35:28.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3b79ffb700 1 -- 172.21.15.66:0/1330270355 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b840041d0 con 0x55f924969db0 2023-12-11T19:35:28.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f92495cbc0 con 0x55f924969db0 2023-12-11T19:35:28.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3b79ffb700 1 -- 172.21.15.66:0/1330270355 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b8400f460 con 0x55f924969db0 2023-12-11T19:35:28.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3bac692b80 10 client.25805 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:28.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f9246e6e60 con 0x55f924969db0 2023-12-11T19:35:28.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.735+0000 7f3b79ffb700 1 -- 172.21.15.66:0/1330270355 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f92495cbc0 con 0x55f924969db0 2023-12-11T19:35:28.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3b79ffb700 1 -- 172.21.15.66:0/1330270355 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3b840244e0 con 0x55f924969db0 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3bac692b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3bac692b80 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3bac692b80 20 client.25805 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f92497f120 0x55f924981540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3bac692b80 4 client.25805 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f924938fc0 con 0x55f92497f120 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.736+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f92497f120 0x55f924981540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.737+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f92497f120 0x55f924981540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fde2d8fc78b3426 server_cookie=dc8c7814a210e7ca in_seq=0 out_seq=0 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.737+0000 7f3b79ffb700 10 client.25805 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.737+0000 7f3b79ffb700 1 -- 172.21.15.66:0/1330270355 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f924938fc0 con 0x55f92497f120 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.737+0000 7f3b79ffb700 10 client.25805 handle_command_reply: tid=0 2023-12-11T19:35:28.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 client.25805 shutdown 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 msgr2=0x55f924973830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 0x55f924973830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b84003f60 tx=0x7f3b84003c60 comp rx=0 tx=0).stop 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 shutdown_connections 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f92497f120 0x55f924981540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3b79ffb700 0 client.25805 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 --2- 172.21.15.66:0/1330270355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 >> 172.21.15.66:0/1330270355 conn(0x55f924898860 msgr2=0x55f924964ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:28.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 shutdown_connections 2023-12-11T19:35:28.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 1 -- 172.21.15.66:0/1330270355 wait complete. 2023-12-11T19:35:28.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.738+0000 7f3bac692b80 20 client.25805 trim_cache size 0 max 16384 2023-12-11T19:35:28.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.757+0000 7f3bac692b80 1 Processor -- start 2023-12-11T19:35:28.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.757+0000 7f3bac692b80 1 -- start start 2023-12-11T19:35:28.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3bac692b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92497c5d0 con 0x55f92496aca0 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3bac692b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f924986750 con 0x55f924969450 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3bac692b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f924968ff0 con 0x55f924969db0 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3b9259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3b9259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37402/0 (socket says 172.21.15.66:37402) 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3b9259c700 1 -- 172.21.15.66:0/3570501447 learned_addr learned my addr 172.21.15.66:0/3570501447 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:28.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.758+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.759+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.759+0000 7f3b9259c700 1 -- 172.21.15.66:0/3570501447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 msgr2=0x55f9249732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.759+0000 7f3b9259c700 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.759+0000 7f3b9259c700 1 -- 172.21.15.66:0/3570501447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 msgr2=0x55f92497a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.759+0000 7f3b9259c700 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.759+0000 7f3b9259c700 1 -- 172.21.15.66:0/3570501447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9246cfce0 con 0x55f924969db0 2023-12-11T19:35:28.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.760+0000 7f3b9259c700 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f924973830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b840098e0 tx=0x7f3b84005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc0c86c4c4cf72d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:28.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.760+0000 7f3b817fa700 1 -- 172.21.15.66:0/3570501447 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401d070 con 0x55f924969db0 2023-12-11T19:35:28.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.760+0000 7f3b817fa700 1 -- 172.21.15.66:0/3570501447 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b8401e040 con 0x55f924969db0 2023-12-11T19:35:28.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.760+0000 7f3b817fa700 1 -- 172.21.15.66:0/3570501447 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b840246b0 con 0x55f924969db0 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 -- 172.21.15.66:0/3570501447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 msgr2=0x55f924973830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f924973830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b840098e0 tx=0x7f3b84005cd0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 -- 172.21.15.66:0/3570501447 shutdown_connections 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f924969450 0x55f9249732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f92496aca0 0x55f92497a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 --2- 172.21.15.66:0/3570501447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f924973830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 -- 172.21.15.66:0/3570501447 >> 172.21.15.66:0/3570501447 conn(0x55f924898860 msgr2=0x55f92496d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.761+0000 7f3bac692b80 1 -- 172.21.15.66:0/3570501447 shutdown_connections 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.762+0000 7f3bac692b80 1 -- 172.21.15.66:0/3570501447 wait complete. 2023-12-11T19:35:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.762+0000 7f3bac692b80 1 -- start start 2023-12-11T19:35:28.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.762+0000 7f3bac692b80 1 Processor -- start 2023-12-11T19:35:28.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 0x55f92497a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92497a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f92496aca0 0x55f924976db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9249774c0 con 0x55f924969450 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f924977630 con 0x55f92496aca0 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9249777f0 con 0x55f924969db0 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3bac692b80 10 client.0 _ll_register_callbacks cb 0x55f9247a2b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3b91d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 0x55f92497a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3b91d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 0x55f92497a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51998/0 (socket says 172.21.15.66:51998) 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3b91d9b700 1 -- 172.21.15.66:0/2605809871 learned_addr learned my addr 172.21.15.66:0/2605809871 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3b9259c700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92497a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.764+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f92496aca0 0x55f924976db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b91d9b700 1 -- 172.21.15.66:0/2605809871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 msgr2=0x55f92497a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92497a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b91d9b700 1 -- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f92496aca0 msgr2=0x55f924976db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f92496aca0 0x55f924976db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b91d9b700 1 -- 172.21.15.66:0/2605809871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9246e6e60 con 0x55f924969450 2023-12-11T19:35:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b91d9b700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 0x55f92497a290 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009a00 tx=0x7f3b88021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a76dd7439cc9c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b79ffb700 1 -- 172.21.15.66:0/2605809871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88035070 con 0x55f924969450 2023-12-11T19:35:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3b79ffb700 1 -- 172.21.15.66:0/2605809871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b88030610 con 0x55f924969450 2023-12-11T19:35:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.765+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b88004830 con 0x55f924969450 2023-12-11T19:35:28.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.766+0000 7f3b79ffb700 1 -- 172.21.15.66:0/2605809871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b88024070 con 0x55f924969450 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.766+0000 7f3bac692b80 10 client.16080 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.766+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f92470d2d0 con 0x55f924969450 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.766+0000 7f3b79ffb700 1 -- 172.21.15.66:0/2605809871 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b88004830 con 0x55f924969450 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.766+0000 7f3b79ffb700 1 -- 172.21.15.66:0/2605809871 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f3b8801d070 con 0x55f924969450 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3bac692b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3bac692b80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3bac692b80 20 client.16080 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3bac692b80 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f9249d6b70 0x55f9249d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3bac692b80 4 client.16080 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f924884050 con 0x55f9249d6b70 2023-12-11T19:35:28.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.767+0000 7f3b9259c700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f9249d6b70 0x55f9249d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.768+0000 7f3b9259c700 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f9249d6b70 0x55f9249d8f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85b8c80035ed6e2 server_cookie=54ff1ac29c70a181 in_seq=0 out_seq=0 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.768+0000 7f3b79ffb700 10 client.16080 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.769+0000 7f3b79ffb700 1 -- 172.21.15.66:0/2605809871 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f924884050 con 0x55f9249d6b70 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.769+0000 7f3b79ffb700 10 client.16080 handle_command_reply: tid=0 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.769+0000 7f3bac692b80 1 client.16080 shutdown 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.769+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 msgr2=0x55f92497a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.769+0000 7f3bac692b80 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 0x55f92497a290 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009a00 tx=0x7f3b88021040 comp rx=0 tx=0).stop 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 shutdown_connections 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55f9249d6b70 0x55f9249d8f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f92496aca0 0x55f924976db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f924969450 0x55f92497a290 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3b79ffb700 0 client.16080 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 --2- 172.21.15.66:0/2605809871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f924969db0 0x55f92497a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 >> 172.21.15.66:0/2605809871 conn(0x55f924898860 msgr2=0x55f9249659f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 shutdown_connections 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 1 -- 172.21.15.66:0/2605809871 wait complete. 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.770+0000 7f3bac692b80 20 client.16080 trim_cache size 0 max 16384 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "5905cda5-0a05-489a-bd64-fd4b4ec98e24", 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-12-11T19:35:28.772 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b7c0a14d0 msgr2=0x7f3b7c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b7c0a14d0 0x7f3b7c0a3990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009c10 tx=0x7f3b8c004bf0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 msgr2=0x7f3b941a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 0x7f3b941a4f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ef30 tx=0x7f3b8c00c3a0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 shutdown_connections 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3b7c0a14d0 0x7f3b7c0a3990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b940ffdf0 0x7f3b9419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b94105130 0x7f3b941a4f90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 --2- 172.21.15.66:0/502608183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b94102760 0x7f3b9419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 >> 172.21.15.66:0/502608183 conn(0x7f3b940fb9f0 msgr2=0x7f3b941032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.771+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 shutdown_connections 2023-12-11T19:35:28.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:28.772+0000 7f3b98dff700 1 -- 172.21.15.66:0/502608183 wait complete. 2023-12-11T19:35:28.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:35:29.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 -- 172.21.15.66:0/3321517554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 msgr2=0x7f42a4104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 --2- 172.21.15.66:0/3321517554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4104c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4294009920 tx=0x7f4294009c30 comp rx=0 tx=0).stop 2023-12-11T19:35:29.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 -- 172.21.15.66:0/3321517554 shutdown_connections 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 --2- 172.21.15.66:0/3321517554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 0x7f42a410ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 --2- 172.21.15.66:0/3321517554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a4103610 0x7f42a4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 --2- 172.21.15.66:0/3321517554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4104c60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 -- 172.21.15.66:0/3321517554 >> 172.21.15.66:0/3321517554 conn(0x7f42a40feb60 msgr2=0x7f42a4100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 -- 172.21.15.66:0/3321517554 shutdown_connections 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.034+0000 7f42a919d700 1 -- 172.21.15.66:0/3321517554 wait complete. 2023-12-11T19:35:29.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.036+0000 7f42a919d700 1 Processor -- start 2023-12-11T19:35:29.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.036+0000 7f42a919d700 1 -- start start 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.036+0000 7f42a919d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a4103610 0x7f42a4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a919d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a919d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 0x7f42a41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a919d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a41a1080 con 0x7f42a41051a0 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a919d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a41a11c0 con 0x7f42a4104800 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a919d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a41a1350 con 0x7f42a4103610 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a4103610 0x7f42a4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37222/0 (socket says 172.21.15.66:37222) 2023-12-11T19:35:29.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 -- 172.21.15.66:0/2743177201 learned_addr learned my addr 172.21.15.66:0/2743177201 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:29.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a899c700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 0x7f42a41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 -- 172.21.15.66:0/2743177201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a4103610 msgr2=0x7f42a4199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a4103610 0x7f42a4199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 -- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 msgr2=0x7f42a41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 0x7f42a41a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a37fe700 1 -- 172.21.15.66:0/2743177201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009d00 con 0x7f42a4104800 2023-12-11T19:35:29.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.037+0000 7f42a899c700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 0x7f42a41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.038+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4199a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4294009840 tx=0x7f429400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.038+0000 7f42a17fa700 1 -- 172.21.15.66:0/2743177201 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429401c070 con 0x7f42a4104800 2023-12-11T19:35:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.038+0000 7f42a17fa700 1 -- 172.21.15.66:0/2743177201 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f429400f7d0 con 0x7f42a4104800 2023-12-11T19:35:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.038+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4294009520 con 0x7f42a4104800 2023-12-11T19:35:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.038+0000 7f42a17fa700 1 -- 172.21.15.66:0/2743177201 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294003ae0 con 0x7f42a4104800 2023-12-11T19:35:29.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.038+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a41a1810 con 0x7f42a4104800 2023-12-11T19:35:29.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.042+0000 7f42a17fa700 1 -- 172.21.15.66:0/2743177201 <== mon.2 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4294005040 con 0x7f42a4104800 2023-12-11T19:35:29.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.044+0000 7f42a17fa700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42900a15d0 0x7f42900a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.044+0000 7f42a17fa700 1 -- 172.21.15.66:0/2743177201 <== mon.2 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42940cdcd0 con 0x7f42a4104800 2023-12-11T19:35:29.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.045+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42900a15d0 0x7f42900a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.045+0000 7f42bdd8db80 1 Processor -- start 2023-12-11T19:35:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 -- start start 2023-12-11T19:35:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42900a15d0 0x7f42900a3a90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42a4104660 tx=0x7f428c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88398380 0x557c883987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397040 0x557c883974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c8839f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c882e9b50 con 0x557c88397a00 2023-12-11T19:35:29.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c882e7470 con 0x557c88398380 2023-12-11T19:35:29.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.046+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c882e2d60 con 0x557c88397040 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.047+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88398380 0x557c883987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.047+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88398380 0x557c883987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37224/0 (socket says 172.21.15.66:37224) 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.047+0000 7f42a3fff700 1 -- 172.21.15.66:0/2197837348 learned_addr learned my addr 172.21.15.66:0/2197837348 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.047+0000 7f42a899c700 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c8839f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.047+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397040 0x557c883974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f42a899c700 1 -- 172.21.15.66:0/2197837348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397040 msgr2=0x557c883974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f42a899c700 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397040 0x557c883974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f42a899c700 1 -- 172.21.15.66:0/2197837348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88398380 msgr2=0x557c883987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f42a899c700 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88398380 0x557c883987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f42a899c700 1 -- 172.21.15.66:0/2197837348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8815f630 con 0x557c88397a00 2023-12-11T19:35:29.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f42a899c700 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c8839f2a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f429c009500 tx=0x7f429c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f36870b0d170a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.048+0000 7f429a7fc700 1 -- 172.21.15.66:0/2197837348 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00e030 con 0x557c88397a00 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.049+0000 7f429a7fc700 1 -- 172.21.15.66:0/2197837348 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f429c00f040 con 0x557c88397a00 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.049+0000 7f429a7fc700 1 -- 172.21.15.66:0/2197837348 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c015c00 con 0x557c88397a00 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.049+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2197837348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 msgr2=0x557c8839f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.049+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c8839f2a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f429c009500 tx=0x7f429c0099e0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2197837348 shutdown_connections 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88398380 0x557c883987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c8839f2a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2197837348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397040 0x557c883974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2197837348 >> 172.21.15.66:0/2197837348 conn(0x557c882bd7e0 msgr2=0x557c883893d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2197837348 shutdown_connections 2023-12-11T19:35:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.050+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2197837348 wait complete. 2023-12-11T19:35:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 -- start start 2023-12-11T19:35:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 Processor -- start 2023-12-11T19:35:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397040 0x557c883a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397a00 0x557c883a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8829c560 con 0x557c88397040 2023-12-11T19:35:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8829c200 con 0x557c88397a00 2023-12-11T19:35:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.051+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c88161920 con 0x557c88398380 2023-12-11T19:35:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42bdd8db80 10 client.0 _ll_register_callbacks cb 0x557c881c7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a899c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397a00 0x557c883a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a899c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37456/0 (socket says 172.21.15.66:37456) 2023-12-11T19:35:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a899c700 1 -- 172.21.15.66:0/2006147953 learned_addr learned my addr 172.21.15.66:0/2006147953 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397040 0x557c883a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a37fe700 1 -- 172.21.15.66:0/2006147953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 msgr2=0x557c883a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.052+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42a37fe700 1 -- 172.21.15.66:0/2006147953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397040 msgr2=0x557c883a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397040 0x557c883a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42a37fe700 1 -- 172.21.15.66:0/2006147953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c0091b0 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397a00 0x557c883a1110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f429400f9d0 tx=0x7f429400bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3496da5065530603 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42837fe700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42837fe700 1 -- 172.21.15.66:0/2006147953 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429409b4e0 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c883819e0 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42837fe700 1 -- 172.21.15.66:0/2006147953 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f429409f1d0 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42837fe700 1 -- 172.21.15.66:0/2006147953 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42940a3380 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42bdd8db80 10 client.25823 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.053+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c8810bde0 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42837fe700 1 -- 172.21.15.66:0/2006147953 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c883819e0 con 0x557c88397a00 2023-12-11T19:35:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42837fe700 1 -- 172.21.15.66:0/2006147953 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f42940a2ef0 con 0x557c88397a00 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42bdd8db80 10 client.25823 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42bdd8db80 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42bdd8db80 20 client.25823 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c883ac570 0x557c883ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42bdd8db80 4 client.25823 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.054+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c8835df40 con 0x557c883ac570 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.055+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c883ac570 0x557c883ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.055+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c883ac570 0x557c883ae990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491be5066843d8f8 server_cookie=4ea8222ec61cdd0 in_seq=0 out_seq=0 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.055+0000 7f42837fe700 10 client.25823 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42837fe700 1 -- 172.21.15.66:0/2006147953 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c8835df40 con 0x557c883ac570 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42837fe700 10 client.25823 handle_command_reply: tid=0 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 client.25823 shutdown 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397a00 msgr2=0x557c883a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397a00 0x557c883a1110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f429400f9d0 tx=0x7f429400bf80 comp rx=0 tx=0).stop 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 shutdown_connections 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c883ac570 0x557c883ae990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397040 0x557c883a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42837fe700 0 client.25823 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2006147953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397a00 0x557c883a1110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.056+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 >> 172.21.15.66:0/2006147953 conn(0x557c882bd7e0 msgr2=0x557c8838d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.057+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 shutdown_connections 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.057+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2006147953 wait complete. 2023-12-11T19:35:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.057+0000 7f42bdd8db80 20 client.25823 trim_cache size 0 max 16384 2023-12-11T19:35:29.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 Processor -- start 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 -- start start 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397a00 0x557c883a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88398380 0x557c883893a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8835df40 con 0x557c88398380 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883819e0 con 0x557c88397040 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.081+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883a0740 con 0x557c88397a00 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a899c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88398380 0x557c883893a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37244/0 (socket says 172.21.15.66:37244) 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 -- 172.21.15.66:0/2331765313 learned_addr learned my addr 172.21.15.66:0/2331765313 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:29.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a37fe700 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397a00 0x557c883a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 -- 172.21.15.66:0/2331765313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397a00 msgr2=0x557c883a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397a00 0x557c883a8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 -- 172.21.15.66:0/2331765313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88398380 msgr2=0x557c883893a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88398380 0x557c883893a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 -- 172.21.15.66:0/2331765313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c880f4c60 con 0x557c88397040 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.082+0000 7f42a3fff700 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a00e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x557c883af2c0 tx=0x7f428c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b092ffec4392329 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:29.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f429affd700 1 -- 172.21.15.66:0/2331765313 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c01db00 con 0x557c88397040 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f429affd700 1 -- 172.21.15.66:0/2331765313 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f428c023040 con 0x557c88397040 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f429affd700 1 -- 172.21.15.66:0/2331765313 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c003b40 con 0x557c88397040 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2331765313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 msgr2=0x557c883a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a00e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x557c883af2c0 tx=0x7f428c014000 comp rx=0 tx=0).stop 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2331765313 shutdown_connections 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a00e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.083+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88398380 0x557c883893a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.084+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/2331765313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88397a00 0x557c883a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.084+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2331765313 >> 172.21.15.66:0/2331765313 conn(0x557c882bd7e0 msgr2=0x557c883b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.084+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2331765313 shutdown_connections 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.084+0000 7f42bdd8db80 1 -- 172.21.15.66:0/2331765313 wait complete. 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.084+0000 7f42bdd8db80 1 -- start start 2023-12-11T19:35:29.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.084+0000 7f42bdd8db80 1 Processor -- start 2023-12-11T19:35:29.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c883a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883a6f10 con 0x557c88397a00 2023-12-11T19:35:29.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883a7080 con 0x557c88397040 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883a71f0 con 0x557c88398380 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42bdd8db80 10 client.0 _ll_register_callbacks cb 0x557c881c7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c883a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42a899c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c883a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52040/0 (socket says 172.21.15.66:52040) 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42a37fe700 1 -- 172.21.15.66:0/73214138 learned_addr learned my addr 172.21.15.66:0/73214138 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.085+0000 7f42a3fff700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a37fe700 1 -- 172.21.15.66:0/73214138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 msgr2=0x557c883a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a37fe700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a37fe700 1 -- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 msgr2=0x557c883a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a37fe700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a37fe700 1 -- 172.21.15.66:0/73214138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c017070 con 0x557c88397a00 2023-12-11T19:35:29.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a3fff700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42a37fe700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c883a6490 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f42940094f0 tx=0x7f4294009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a5de4f62b5abcf server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42837fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42837fe700 1 -- 172.21.15.66:0/73214138 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429409b4e0 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f429c00f040 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.086+0000 7f42837fe700 1 -- 172.21.15.66:0/73214138 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f429409f1d0 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42837fe700 1 -- 172.21.15.66:0/73214138 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42940b3ab0 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 10 client.16098 fetch_fsmap learned FSMap version 28 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c88132250 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42837fe700 1 -- 172.21.15.66:0/73214138 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f429c00f040 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42837fe700 1 -- 172.21.15.66:0/73214138 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f429409a250 con 0x557c88397a00 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2023-12-11T19:35:29.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 20 client.16098 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c88403cd0 0x557c884060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 4 client.16098 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.087+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c882a8fd0 con 0x557c88403cd0 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.088+0000 7f42a3fff700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c88403cd0 0x557c884060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.088+0000 7f42a3fff700 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c88403cd0 0x557c884060c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97086161f243b2cb server_cookie=154f999ae49d0f51 in_seq=0 out_seq=0 2023-12-11T19:35:29.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.088+0000 7f42837fe700 10 client.16098 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.088+0000 7f42837fe700 1 -- 172.21.15.66:0/73214138 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557c882a8fd0 con 0x557c88403cd0 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.088+0000 7f42837fe700 10 client.16098 handle_command_reply: tid=0 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.088+0000 7f42bdd8db80 1 client.16098 shutdown 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 msgr2=0x557c883a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c883a6490 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f42940094f0 tx=0x7f4294009c30 comp rx=0 tx=0).stop 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 shutdown_connections 2023-12-11T19:35:29.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x557c88403cd0 0x557c884060c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557c88397040 0x557c883a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c88398380 0x557c883a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 --2- 172.21.15.66:0/73214138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c88397a00 0x557c883a6490 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42837fe700 0 client.16098 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 >> 172.21.15.66:0/73214138 conn(0x557c882bd7e0 msgr2=0x557c883b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 shutdown_connections 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 1 -- 172.21.15.66:0/73214138 wait complete. 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.089+0000 7f42bdd8db80 20 client.16098 trim_cache size 0 max 16384 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:35:29.094 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: "5905cda5-0a05-489a-bd64-fd4b4ec98e24": { 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "5905cda5-0a05-489a-bd64-fd4b4ec98e24", 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:35:29.095 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42900a15d0 msgr2=0x7f42900a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42900a15d0 0x7f42900a3a90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42a4104660 tx=0x7f428c009260 comp rx=0 tx=0).stop 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 msgr2=0x7f42a4199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4199a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4294009840 tx=0x7f429400b790 comp rx=0 tx=0).stop 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 shutdown_connections 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42900a15d0 0x7f42900a3a90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f42a41051a0 0x7f42a41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42a4103610 0x7f42a4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 --2- 172.21.15.66:0/2743177201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42a4104800 0x7f42a4199a90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 >> 172.21.15.66:0/2743177201 conn(0x7f42a40feb60 msgr2=0x7f42a4107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 shutdown_connections 2023-12-11T19:35:29.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:29.090+0000 7f42a919d700 1 -- 172.21.15.66:0/2743177201 wait complete. 2023-12-11T19:35:29.109 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:35:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-k[139159]: 2023-12-11T19:35:28.772+0000 7fadedc07700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [2,head] /volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106/client.0/tmp/f1 auth v32 ap=1 s=1073741824 n(v0 rc2023-12-11T19:35:16.176747+0000 b1073741824 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-2151677952@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@4},l=15381 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559adb652100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000007:[<0x10000000006/f1 v26>,<0x10000000005/tmp v28>,<0x10000000003/client.0 v16>,<0x10000000002/d7198378-99a5-4b89-915f-9be6900c3106 v32>,<0x10000000001/sv_0 v55>,<0x10000000000/_nogroup v41>,<0x1/volumes v41>]//[]","memoryvalue":"(3)0x10000000007:[<0x10000000006/f1 v32>,<0x10000000005/tmp v33>,<0x10000000003/client.0 v22>,<0x10000000002/d7198378-99a5-4b89-915f-9be6900c3106 v39>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v23>,<0x1/volumes v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-11T19:35:29.111 INFO:tasks.cephfs.filesystem:scrub status for tag:5905cda5-0a05-489a-bd64-fd4b4ec98e24 - {'path': '/', 'tag': '5905cda5-0a05-489a-bd64-fd4b4ec98e24', 'options': 'recursive,force'} 2023-12-11T19:35:29.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: pgmap v417: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 35 MiB/s wr, 20 op/s 2023-12-11T19:35:29.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: scrub queued for path: / 2023-12-11T19:35:29.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:29.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: scrub summary: active paths [/] 2023-12-11T19:35:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: scrub summary: active paths [/] 2023-12-11T19:35:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: Scrub error on inode 0x10000000007 (/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106/client.0/tmp/f1) see mds.k log and `damage ls` output for details 2023-12-11T19:35:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:29.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:29 smithi070 ceph-mon[104044]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: pgmap v417: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 35 MiB/s wr, 20 op/s 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: scrub queued for path: / 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: scrub summary: active paths [/] 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: scrub summary: active paths [/] 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: Scrub error on inode 0x10000000007 (/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106/client.0/tmp/f1) see mds.k log and `damage ls` output for details 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:30.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:29 smithi167 ceph-mon[102750]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: pgmap v417: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 35 MiB/s wr, 20 op/s 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: scrub queued for path: / 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: scrub summary: active paths [/] 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: scrub summary: active paths [/] 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: Scrub error on inode 0x10000000007 (/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106/client.0/tmp/f1) see mds.k log and `damage ls` output for details 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:29 smithi066 ceph-mon[97920]: scrub summary: idle+waiting paths [/] 2023-12-11T19:35:31.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:30 smithi167 ceph-mon[102750]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-11T19:35:31.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:30 smithi167 ceph-mon[102750]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:active 2023-12-11T19:35:31.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:30 smithi167 ceph-mon[102750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:31.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:30 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a[97896]: 2023-12-11T19:35:30.587+0000 7efd84ddd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-11T19:35:31.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:30 smithi066 ceph-mon[97920]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-11T19:35:31.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:30 smithi066 ceph-mon[97920]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:active 2023-12-11T19:35:31.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:30 smithi066 ceph-mon[97920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:31.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:30 smithi070 ceph-mon[104044]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-11T19:35:31.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:30 smithi070 ceph-mon[104044]: mds.? [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] up:active 2023-12-11T19:35:31.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:30 smithi070 ceph-mon[104044]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 -- 172.21.15.66:0/1100547642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 msgr2=0x7f2550104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 --2- 172.21.15.66:0/1100547642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550104da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098f0 tx=0x7f253c009c00 comp rx=0 tx=0).stop 2023-12-11T19:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 -- 172.21.15.66:0/1100547642 shutdown_connections 2023-12-11T19:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 --2- 172.21.15.66:0/1100547642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2550103750 0x7f2550103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 --2- 172.21.15.66:0/1100547642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25501052e0 0x7f255010bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.907 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 --2- 172.21.15.66:0/1100547642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550104da0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.898+0000 7f2556bff700 1 -- 172.21.15.66:0/1100547642 >> 172.21.15.66:0/1100547642 conn(0x7f25500fece0 msgr2=0x7f2550101100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.899+0000 7f2556bff700 1 -- 172.21.15.66:0/1100547642 shutdown_connections 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.899+0000 7f2556bff700 1 -- 172.21.15.66:0/1100547642 wait complete. 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 Processor -- start 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 -- start start 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2550103750 0x7f2550199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25501052e0 0x7f25501a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501a1130 con 0x7f25501052e0 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501a1270 con 0x7f2550103750 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.900+0000 7f2556bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501a1400 con 0x7f2550104940 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f255519c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25501052e0 0x7f25501a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.908 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37506/0 (socket says 172.21.15.66:37506) 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 -- 172.21.15.66:0/671594456 learned_addr learned my addr 172.21.15.66:0/671594456 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f255499b700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2550103750 0x7f2550199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 -- 172.21.15.66:0/671594456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2550103750 msgr2=0x7f2550199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2550103750 0x7f2550199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 -- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25501052e0 msgr2=0x7f25501a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25501052e0 0x7f25501a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 -- 172.21.15.66:0/671594456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009d00 con 0x7f2550104940 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.901+0000 7f254ffff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550199b90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f253c009810 tx=0x7f253c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.902+0000 7f254dffb700 1 -- 172.21.15.66:0/671594456 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c00f7d0 con 0x7f2550104940 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.902+0000 7f254dffb700 1 -- 172.21.15.66:0/671594456 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f253c0054d0 con 0x7f2550104940 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.902+0000 7f254dffb700 1 -- 172.21.15.66:0/671594456 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c003dc0 con 0x7f2550104940 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.902+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c009520 con 0x7f2550104940 2023-12-11T19:35:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.902+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a1910 con 0x7f2550104940 2023-12-11T19:35:31.911 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.906+0000 7f254dffb700 1 -- 172.21.15.66:0/671594456 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f253c005040 con 0x7f2550104940 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.907+0000 7f254dffb700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25400a1500 0x7f25400a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.907+0000 7f254dffb700 1 -- 172.21.15.66:0/671594456 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f253c0cdfa0 con 0x7f2550104940 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.907+0000 7f256a56cb80 1 Processor -- start 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.907+0000 7f255499b700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25400a1500 0x7f25400a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 -- start start 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfaf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfafba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cf09af0 con 0x55a78cfae830 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cf07410 con 0x55a78cfaded0 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f256a56cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cf02d00 con 0x55a78cfaf720 2023-12-11T19:35:31.923 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f255499b700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25400a1500 0x7f25400a39c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f25501047a0 tx=0x7f2544009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f254ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfaf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f255499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.908+0000 7f255499b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:37286/0 (socket says 172.21.15.66:37286) 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 -- 172.21.15.66:0/3600430148 learned_addr learned my addr 172.21.15.66:0/3600430148 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255519c700 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfafba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 -- 172.21.15.66:0/3600430148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 msgr2=0x55a78cfafba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfafba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 -- 172.21.15.66:0/3600430148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 msgr2=0x55a78cfaf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfaf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 -- 172.21.15.66:0/3600430148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78cd7f5d0 con 0x55a78cfaded0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.909+0000 7f255499b700 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2544011d20 tx=0x7f2544014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577b9ef58c15358 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.910+0000 7f253affd700 1 -- 172.21.15.66:0/3600430148 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400ed20 con 0x55a78cfaded0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.910+0000 7f253affd700 1 -- 172.21.15.66:0/3600430148 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2544021070 con 0x55a78cfaded0 2023-12-11T19:35:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.910+0000 7f253affd700 1 -- 172.21.15.66:0/3600430148 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544025630 con 0x55a78cfaded0 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 -- 172.21.15.66:0/3600430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 msgr2=0x55a78cfae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2544011d20 tx=0x7f2544014000 comp rx=0 tx=0).stop 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 -- 172.21.15.66:0/3600430148 shutdown_connections 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfafba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 --2- 172.21.15.66:0/3600430148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 -- 172.21.15.66:0/3600430148 >> 172.21.15.66:0/3600430148 conn(0x55a78cedd780 msgr2=0x55a78cfaa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 -- 172.21.15.66:0/3600430148 shutdown_connections 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 -- 172.21.15.66:0/3600430148 wait complete. 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.911+0000 7f256a56cb80 1 -- start start 2023-12-11T19:35:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 Processor -- start 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 0x55a78cfb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfb83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78cfbf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cebc500 con 0x55a78cfae830 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cebc1a0 con 0x55a78cfaf720 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cd818c0 con 0x55a78cfaded0 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f256a56cb80 10 client.0 _ll_register_callbacks cb 0x55a78cde7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f255499b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 0x55a78cfb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f254ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfb83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f255499b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 0x55a78cfb7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:37514/0 (socket says 172.21.15.66:37514) 2023-12-11T19:35:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f255499b700 1 -- 172.21.15.66:0/2094674009 learned_addr learned my addr 172.21.15.66:0/2094674009 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.912+0000 7f255519c700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78cfbf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f254ffff700 1 -- 172.21.15.66:0/2094674009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 msgr2=0x55a78cfb7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f254ffff700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 0x55a78cfb7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f254ffff700 1 -- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 msgr2=0x55a78cfbf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f254ffff700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78cfbf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f254ffff700 1 -- 172.21.15.66:0/2094674009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2544017070 con 0x55a78cfae830 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f255499b700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 0x55a78cfb7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f254ffff700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfb83f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098c0 tx=0x7f253c09af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b0e663d50acf3 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f252ffff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f252ffff700 1 -- 172.21.15.66:0/2094674009 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c09ba10 con 0x55a78cfae830 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a78cfa1730 con 0x55a78cfae830 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.913+0000 7f252ffff700 1 -- 172.21.15.66:0/2094674009 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f253c09b310 con 0x55a78cfae830 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f252ffff700 1 -- 172.21.15.66:0/2094674009 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f253c09f460 con 0x55a78cfae830 2023-12-11T19:35:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 10 client.16110 fetch_fsmap learned FSMap version 29 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a78cd2bd80 con 0x55a78cfae830 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f252ffff700 1 -- 172.21.15.66:0/2094674009 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a78cfa1730 con 0x55a78cfae830 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f252ffff700 1 -- 172.21.15.66:0/2094674009 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f253c09b570 con 0x55a78cfae830 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 20 client.16110 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78cfc3830 0x55a78cfc5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 4 client.16110 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.914+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78cf7dee0 con 0x55a78cfc3830 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.915+0000 7f255499b700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78cfc3830 0x55a78cfc5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.915+0000 7f255499b700 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78cfc3830 0x55a78cfc5c50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48aeb1bae7736235 server_cookie=b9583a12393b0c28 in_seq=0 out_seq=0 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.915+0000 7f252ffff700 10 client.16110 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f252ffff700 1 -- 172.21.15.66:0/2094674009 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a78cf7dee0 con 0x55a78cfc3830 2023-12-11T19:35:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f252ffff700 10 client.16110 handle_command_reply: tid=0 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f256a56cb80 1 client.16110 shutdown 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 msgr2=0x55a78cfb83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f256a56cb80 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfb83f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098c0 tx=0x7f253c09af20 comp rx=0 tx=0).stop 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 shutdown_connections 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.916+0000 7f256a56cb80 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78cfc3830 0x55a78cfc5c50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78cfbf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaded0 0x55a78cfb7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 1 --2- 172.21.15.66:0/2094674009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfb83f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 >> 172.21.15.66:0/2094674009 conn(0x55a78cedd780 msgr2=0x55a78cfa8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f252ffff700 0 client.16110 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 shutdown_connections 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 1 -- 172.21.15.66:0/2094674009 wait complete. 2023-12-11T19:35:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.917+0000 7f256a56cb80 20 client.16110 trim_cache size 0 max 16384 2023-12-11T19:35:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.943+0000 7f256a56cb80 1 Processor -- start 2023-12-11T19:35:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.943+0000 7f256a56cb80 1 -- start start 2023-12-11T19:35:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfbf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.948 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f256a56cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cfb8e90 con 0x55a78cfae830 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f256a56cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cfaef30 con 0x55a78cfaded0 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f256a56cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78cfb5920 con 0x55a78cfaf720 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f254ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfbf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f254ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfbf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52082/0 (socket says 172.21.15.66:52082) 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f254ffff700 1 -- 172.21.15.66:0/1901666396 learned_addr learned my addr 172.21.15.66:0/1901666396 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.944+0000 7f255519c700 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f254ffff700 1 -- 172.21.15.66:0/1901666396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 msgr2=0x55a78cfca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f254ffff700 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f254ffff700 1 -- 172.21.15.66:0/1901666396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 msgr2=0x55a78cfae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f254ffff700 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f254ffff700 1 -- 172.21.15.66:0/1901666396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78cd14c00 con 0x55a78cfae830 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f254ffff700 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfbf1e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f253c0a30e0 tx=0x7f253c09a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0037e33eab5aba server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f253b7fe700 1 -- 172.21.15.66:0/1901666396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c09ba10 con 0x55a78cfae830 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f253b7fe700 1 -- 172.21.15.66:0/1901666396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f253c09b310 con 0x55a78cfae830 2023-12-11T19:35:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.945+0000 7f253b7fe700 1 -- 172.21.15.66:0/1901666396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c09b570 con 0x55a78cfae830 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 -- 172.21.15.66:0/1901666396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 msgr2=0x55a78cfbf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfbf1e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f253c0a30e0 tx=0x7f253c09a800 comp rx=0 tx=0).stop 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 -- 172.21.15.66:0/1901666396 shutdown_connections 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaded0 0x55a78cfae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfaf720 0x55a78cfca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1901666396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfae830 0x55a78cfbf1e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 -- 172.21.15.66:0/1901666396 >> 172.21.15.66:0/1901666396 conn(0x55a78cedd780 msgr2=0x55a78cfb5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.946+0000 7f256a56cb80 1 -- 172.21.15.66:0/1901666396 shutdown_connections 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.947+0000 7f256a56cb80 1 -- 172.21.15.66:0/1901666396 wait complete. 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.947+0000 7f256a56cb80 1 -- start start 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.947+0000 7f256a56cb80 1 Processor -- start 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 0x55a78cfbc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfae830 0x55a78cfbc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78d017d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d018250 con 0x55a78cfaded0 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d018390 con 0x55a78cfaf720 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78d0294e0 con 0x55a78cfae830 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f256a56cb80 10 client.0 _ll_register_callbacks cb 0x55a78cde7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f255499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 0x55a78cfbc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f255499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 0x55a78cfbc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52098/0 (socket says 172.21.15.66:52098) 2023-12-11T19:35:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.948+0000 7f255499b700 1 -- 172.21.15.66:0/1174900216 learned_addr learned my addr 172.21.15.66:0/1174900216 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f254ffff700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfae830 0x55a78cfbc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f255519c700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78d017d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f255499b700 1 -- 172.21.15.66:0/1174900216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfae830 msgr2=0x55a78cfbc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f255499b700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfae830 0x55a78cfbc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f255499b700 1 -- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 msgr2=0x55a78d017d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f255499b700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78d017d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.949+0000 7f255499b700 1 -- 172.21.15.66:0/1174900216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2544017070 con 0x55a78cfaded0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.950+0000 7f255499b700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 0x55a78cfbc3c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f25440117d0 tx=0x7f254401eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22de8ed7f7aa7b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.950+0000 7f252ffff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.950+0000 7f252ffff700 1 -- 172.21.15.66:0/1174900216 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400eba0 con 0x55a78cfaded0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.951+0000 7f252ffff700 1 -- 172.21.15.66:0/1174900216 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2544005040 con 0x55a78cfaded0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.951+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2544021070 con 0x55a78cfaded0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.951+0000 7f252ffff700 1 -- 172.21.15.66:0/1174900216 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2544005200 con 0x55a78cfaded0 2023-12-11T19:35:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.951+0000 7f256a56cb80 10 client.16116 fetch_fsmap learned FSMap version 29 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.951+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a78cd521f0 con 0x55a78cfaded0 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.951+0000 7f252ffff700 1 -- 172.21.15.66:0/1174900216 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2544044020 con 0x55a78cfaded0 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f252ffff700 1 -- 172.21.15.66:0/1174900216 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f25440351b0 con 0x55a78cfaded0 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f256a56cb80 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f256a56cb80 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f256a56cb80 20 client.16116 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78d02adb0 0x55a78d02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f256a56cb80 4 client.16116 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a78cec8f70 con 0x55a78d02adb0 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.952+0000 7f254ffff700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78d02adb0 0x55a78d02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.953+0000 7f254ffff700 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78d02adb0 0x55a78d02d1d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9b72fa04c7ea8f server_cookie=24f1fd0f02727c88 in_seq=0 out_seq=0 2023-12-11T19:35:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.953+0000 7f252ffff700 10 client.16116 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f252ffff700 1 -- 172.21.15.66:0/1174900216 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a78cec8f70 con 0x55a78d02adb0 2023-12-11T19:35:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f252ffff700 10 client.16116 handle_command_reply: tid=0 2023-12-11T19:35:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 client.16116 shutdown 2023-12-11T19:35:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 msgr2=0x55a78cfbc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 0x55a78cfbc3c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f25440117d0 tx=0x7f254401eda0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 shutdown_connections 2023-12-11T19:35:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55a78d02adb0 0x55a78d02d1d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55a78cfaf720 0x55a78d017d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a78cfaded0 0x55a78cfbc3c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 --2- 172.21.15.66:0/1174900216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a78cfae830 0x55a78cfbc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 >> 172.21.15.66:0/1174900216 conn(0x55a78cedd780 msgr2=0x55a78cfca5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f252ffff700 0 client.16116 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 shutdown_connections 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.961+0000 7f256a56cb80 1 -- 172.21.15.66:0/1174900216 wait complete. 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f256a56cb80 20 client.16116 trim_cache size 0 max 16384 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:35:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:35:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25400a1500 msgr2=0x7f25400a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25400a1500 0x7f25400a39c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f25501047a0 tx=0x7f2544009260 comp rx=0 tx=0).stop 2023-12-11T19:35:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 msgr2=0x7f2550199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550199b90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f253c009810 tx=0x7f253c00b810 comp rx=0 tx=0).stop 2023-12-11T19:35:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 shutdown_connections 2023-12-11T19:35:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f25400a1500 0x7f25400a39c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2550103750 0x7f2550199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.962+0000 7f2556bff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25501052e0 0x7f25501a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.963+0000 7f2556bff700 1 --2- 172.21.15.66:0/671594456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2550104940 0x7f2550199b90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.963+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 >> 172.21.15.66:0/671594456 conn(0x7f25500fece0 msgr2=0x7f2550107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.963+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 shutdown_connections 2023-12-11T19:35:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:31.963+0000 7f2556bff700 1 -- 172.21.15.66:0/671594456 wait complete. 2023-12-11T19:35:32.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:31 smithi167 ceph-mon[102750]: pgmap v418: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 39 MiB/s wr, 24 op/s 2023-12-11T19:35:32.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:31 smithi066 ceph-mon[97920]: pgmap v418: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 39 MiB/s wr, 24 op/s 2023-12-11T19:35:32.114 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:31 smithi070 ceph-mon[104044]: pgmap v418: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 39 MiB/s wr, 24 op/s 2023-12-11T19:35:34.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:33 smithi167 ceph-mon[102750]: pgmap v419: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 43 MiB/s wr, 21 op/s 2023-12-11T19:35:34.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:33 smithi066 ceph-mon[97920]: pgmap v419: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 43 MiB/s wr, 21 op/s 2023-12-11T19:35:34.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:33 smithi070 ceph-mon[104044]: pgmap v419: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 43 MiB/s wr, 21 op/s 2023-12-11T19:35:36.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:35 smithi167 ceph-mon[102750]: pgmap v420: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 56 MiB/s wr, 27 op/s 2023-12-11T19:35:36.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:35 smithi066 ceph-mon[97920]: pgmap v420: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 56 MiB/s wr, 27 op/s 2023-12-11T19:35:36.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:35 smithi070 ceph-mon[104044]: pgmap v420: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 56 MiB/s wr, 27 op/s 2023-12-11T19:35:37.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:36 smithi167 ceph-mon[102750]: scrub summary: idle 2023-12-11T19:35:37.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:36 smithi167 ceph-mon[102750]: scrub summary: idle 2023-12-11T19:35:37.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:36 smithi066 ceph-mon[97920]: scrub summary: idle 2023-12-11T19:35:37.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:36 smithi066 ceph-mon[97920]: scrub summary: idle 2023-12-11T19:35:37.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:36 smithi070 ceph-mon[104044]: scrub summary: idle 2023-12-11T19:35:37.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:36 smithi070 ceph-mon[104044]: scrub summary: idle 2023-12-11T19:35:37.475 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:37.475 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 rewriters = 63256.76 kB/sec 2023-12-11T19:35:37.476 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 rewriters = 63254.06 kB/sec 2023-12-11T19:35:37.476 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 63256.76 kB/sec 2023-12-11T19:35:37.476 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 63256.76 kB/sec 2023-12-11T19:35:37.476 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 63256.76 kB/sec 2023-12-11T19:35:37.476 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:35:38.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:37 smithi167 ceph-mon[102750]: pgmap v421: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 29 op/s 2023-12-11T19:35:38.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:37 smithi066 ceph-mon[97920]: pgmap v421: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 29 op/s 2023-12-11T19:35:38.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:37 smithi070 ceph-mon[104044]: pgmap v421: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 59 MiB/s wr, 29 op/s 2023-12-11T19:35:40.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:39 smithi167 ceph-mon[102750]: pgmap v422: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 55 MiB/s wr, 26 op/s 2023-12-11T19:35:40.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:39 smithi066 ceph-mon[97920]: pgmap v422: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 55 MiB/s wr, 26 op/s 2023-12-11T19:35:40.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:39 smithi070 ceph-mon[104044]: pgmap v422: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 55 MiB/s wr, 26 op/s 2023-12-11T19:35:41.444 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:41.444 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 readers = 4710494.50 kB/sec 2023-12-11T19:35:41.444 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 readers = 4696641.08 kB/sec 2023-12-11T19:35:41.445 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 4710494.50 kB/sec 2023-12-11T19:35:41.445 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 4710494.50 kB/sec 2023-12-11T19:35:41.445 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 4710494.50 kB/sec 2023-12-11T19:35:41.445 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:35:42.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:41 smithi167 ceph-mon[102750]: pgmap v423: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 58 MiB/s wr, 29 op/s 2023-12-11T19:35:42.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:41 smithi066 ceph-mon[97920]: pgmap v423: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 58 MiB/s wr, 29 op/s 2023-12-11T19:35:42.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:41 smithi070 ceph-mon[104044]: pgmap v423: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 58 MiB/s wr, 29 op/s 2023-12-11T19:35:43.847 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:43.847 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 re-readers = 3395514.75 kB/sec 2023-12-11T19:35:43.847 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 re-readers = 3387243.04 kB/sec 2023-12-11T19:35:43.848 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 3395514.75 kB/sec 2023-12-11T19:35:43.848 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 3395514.75 kB/sec 2023-12-11T19:35:43.848 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 3395514.75 kB/sec 2023-12-11T19:35:43.848 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:35:44.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:43 smithi167 ceph-mon[102750]: pgmap v424: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 49 MiB/s wr, 24 op/s 2023-12-11T19:35:44.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:43 smithi066 ceph-mon[97920]: pgmap v424: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 49 MiB/s wr, 24 op/s 2023-12-11T19:35:44.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:43 smithi070 ceph-mon[104044]: pgmap v424: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 49 MiB/s wr, 24 op/s 2023-12-11T19:35:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.124+0000 7f94905ff700 1 -- 172.21.15.66:0/351980078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94881030c0 msgr2=0x7f9488109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.124+0000 7f94905ff700 1 --2- 172.21.15.66:0/351980078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94881030c0 0x7f9488109970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f948000b0d0 tx=0x7f948000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 -- 172.21.15.66:0/351980078 shutdown_connections 2023-12-11T19:35:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 --2- 172.21.15.66:0/351980078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9488101530 0x7f9488101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 --2- 172.21.15.66:0/351980078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94881030c0 0x7f9488109970 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 --2- 172.21.15.66:0/351980078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488102720 0x7f9488102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 -- 172.21.15.66:0/351980078 >> 172.21.15.66:0/351980078 conn(0x7f94880fcaa0 msgr2=0x7f94880feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 -- 172.21.15.66:0/351980078 shutdown_connections 2023-12-11T19:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.125+0000 7f94905ff700 1 -- 172.21.15.66:0/351980078 wait complete. 2023-12-11T19:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 Processor -- start 2023-12-11T19:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 -- start start 2023-12-11T19:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488101530 0x7f94881073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 0x7f94881078e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94881030c0 0x7f9488103f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94881046d0 con 0x7f9488102720 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488104840 con 0x7f94881030c0 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.126+0000 7f94905ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9488104a00 con 0x7f9488101530 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 0x7f94881078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 0x7f94881078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58296/0 (socket says 172.21.15.66:58296) 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 -- 172.21.15.66:0/3400704465 learned_addr learned my addr 172.21.15.66:0/3400704465 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948e39b700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488101530 0x7f94881073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948eb9c700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94881030c0 0x7f9488103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 -- 172.21.15.66:0/3400704465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488101530 msgr2=0x7f94881073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488101530 0x7f94881073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 -- 172.21.15.66:0/3400704465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94881030c0 msgr2=0x7f9488103f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94881030c0 0x7f9488103f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 -- 172.21.15.66:0/3400704465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009d00 con 0x7f9488102720 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948e39b700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488101530 0x7f94881073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948eb9c700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94881030c0 0x7f9488103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f948db9a700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 0x7f94881078e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f947800d950 tx=0x7f947800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.127+0000 7f947f7fe700 1 -- 172.21.15.66:0/3400704465 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478009b40 con 0x7f9488102720 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.128+0000 7f947f7fe700 1 -- 172.21.15.66:0/3400704465 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9478009ca0 con 0x7f9488102720 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.128+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94881a5c00 con 0x7f9488102720 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.128+0000 7f947f7fe700 1 -- 172.21.15.66:0/3400704465 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947800f620 con 0x7f9488102720 2023-12-11T19:35:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.128+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94881a6130 con 0x7f9488102720 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.131+0000 7f947f7fe700 1 -- 172.21.15.66:0/3400704465 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9478010460 con 0x7f9488102720 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.133+0000 7f94a4007b80 1 Processor -- start 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.133+0000 7f94a4007b80 1 -- start start 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f947f7fe700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f94740a1500 0x7f94740a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f947f7fe700 1 -- 172.21.15.66:0/3400704465 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f94780ccfc0 con 0x7f9488102720 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 0x55ddf08262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0826830 0x55ddf0827140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f948eb9c700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f94740a1500 0x7f94740a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf0827ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f94a4007b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf0781af0 con 0x55ddf0825ed0 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f94a4007b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf077f410 con 0x55ddf0826830 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f94a4007b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf077ad00 con 0x55ddf0827720 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f948e39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 0x55ddf08262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f948e39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 0x55ddf08262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58306/0 (socket says 172.21.15.66:58306) 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f948e39b700 1 -- 172.21.15.66:0/2653056416 learned_addr learned my addr 172.21.15.66:0/2653056416 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.134+0000 7f948db9a700 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0826830 0x55ddf0827140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948e39b700 1 -- 172.21.15.66:0/2653056416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 msgr2=0x55ddf0827ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948e39b700 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf0827ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948e39b700 1 -- 172.21.15.66:0/2653056416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0826830 msgr2=0x55ddf0827140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948e39b700 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0826830 0x55ddf0827140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948e39b700 1 -- 172.21.15.66:0/2653056416 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf05f75d0 con 0x55ddf0825ed0 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948db9a700 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0826830 0x55ddf0827140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948eb9c700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f94740a1500 0x7f94740a39c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9480009cd0 tx=0x7f9480009c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f948e39b700 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 0x55ddf08262f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f948400bd70 tx=0x7f948400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fbe96a9745c232 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f947cbfe700 1 -- 172.21.15.66:0/2653056416 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400fdd0 con 0x55ddf0825ed0 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f947cbfe700 1 -- 172.21.15.66:0/2653056416 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94840044b0 con 0x55ddf0825ed0 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.135+0000 7f947cbfe700 1 -- 172.21.15.66:0/2653056416 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484003e60 con 0x55ddf0825ed0 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 -- 172.21.15.66:0/2653056416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 msgr2=0x55ddf08262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 0x55ddf08262f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f948400bd70 tx=0x7f948400f640 comp rx=0 tx=0).stop 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 -- 172.21.15.66:0/2653056416 shutdown_connections 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf0827ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0825ed0 0x55ddf08262f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 --2- 172.21.15.66:0/2653056416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0826830 0x55ddf0827140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 -- 172.21.15.66:0/2653056416 >> 172.21.15.66:0/2653056416 conn(0x55ddf0755780 msgr2=0x55ddf0822880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 -- 172.21.15.66:0/2653056416 shutdown_connections 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.136+0000 7f94a4007b80 1 -- 172.21.15.66:0/2653056416 wait complete. 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 -- start start 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 Processor -- start 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf082fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf08371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf0734500 con 0x55ddf0827720 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf07341a0 con 0x55ddf0825ed0 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf05f98c0 con 0x55ddf0826830 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.137+0000 7f94a4007b80 10 client.0 _ll_register_callbacks cb 0x55ddf065fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf08371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf08371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58310/0 (socket says 172.21.15.66:58310) 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 -- 172.21.15.66:0/1458571487 learned_addr learned my addr 172.21.15.66:0/1458571487 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948db9a700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948e39b700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf082fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 -- 172.21.15.66:0/1458571487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 msgr2=0x55ddf08303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 -- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 msgr2=0x55ddf082fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf082fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 -- 172.21.15.66:0/1458571487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948400b9a0 con 0x55ddf0827720 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f948eb9c700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf08371f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9480015000 tx=0x7f94800099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45aadc9c924928cc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.138+0000 7f9471ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f9471ffb700 1 -- 172.21.15.66:0/1458571487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000e070 con 0x55ddf0827720 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f9471ffb700 1 -- 172.21.15.66:0/1458571487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f94800077d0 con 0x55ddf0827720 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddf0819730 con 0x55ddf0827720 2023-12-11T19:35:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f9471ffb700 1 -- 172.21.15.66:0/1458571487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f948001dcf0 con 0x55ddf0827720 2023-12-11T19:35:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f94a4007b80 10 client.16128 fetch_fsmap learned FSMap version 29 2023-12-11T19:35:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ddf05a3d80 con 0x55ddf0827720 2023-12-11T19:35:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f9471ffb700 1 -- 172.21.15.66:0/1458571487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ddf0819730 con 0x55ddf0827720 2023-12-11T19:35:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f9471ffb700 1 -- 172.21.15.66:0/1458571487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f948002ad20 con 0x55ddf0827720 2023-12-11T19:35:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f94a4007b80 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2023-12-11T19:35:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f94a4007b80 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.139+0000 7f94a4007b80 20 client.16128 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.140+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf083bcc0 0x55ddf083e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.140+0000 7f94a4007b80 4 client.16128 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.140+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddf07f5ee0 con 0x55ddf083bcc0 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.140+0000 7f948e39b700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf083bcc0 0x55ddf083e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.141+0000 7f948e39b700 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf083bcc0 0x55ddf083e0e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8766ca46bd7660c server_cookie=e980c43063a1492c in_seq=0 out_seq=0 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.141+0000 7f9471ffb700 10 client.16128 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f9471ffb700 1 -- 172.21.15.66:0/1458571487 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ddf07f5ee0 con 0x55ddf083bcc0 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f9471ffb700 10 client.16128 handle_command_reply: tid=0 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 client.16128 shutdown 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 msgr2=0x55ddf08371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf08371f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9480015000 tx=0x7f94800099e0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 shutdown_connections 2023-12-11T19:35:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf083bcc0 0x55ddf083e0e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf082fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf08371f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f9471ffb700 0 client.16128 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1458571487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.142+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 >> 172.21.15.66:0/1458571487 conn(0x55ddf0755780 msgr2=0x55ddf0820ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.143+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 shutdown_connections 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.143+0000 7f94a4007b80 1 -- 172.21.15.66:0/1458571487 wait complete. 2023-12-11T19:35:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.143+0000 7f94a4007b80 20 client.16128 trim_cache size 0 max 16384 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 Processor -- start 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 -- start start 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0837ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 0x55ddf08371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf082f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf08377f0 con 0x55ddf0826830 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf0826670 con 0x55ddf0825ed0 2023-12-11T19:35:45.159 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f94a4007b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf08432f0 con 0x55ddf0827720 2023-12-11T19:35:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f948db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 0x55ddf08371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.159+0000 7f948e39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0837ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948db9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 0x55ddf08371b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58326/0 (socket says 172.21.15.66:58326) 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948db9a700 1 -- 172.21.15.66:0/1170463733 learned_addr learned my addr 172.21.15.66:0/1170463733 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948eb9c700 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf082f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948e39b700 1 -- 172.21.15.66:0/1170463733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 msgr2=0x55ddf082f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948e39b700 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf082f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948e39b700 1 -- 172.21.15.66:0/1170463733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 msgr2=0x55ddf08371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948e39b700 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 0x55ddf08371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948e39b700 1 -- 172.21.15.66:0/1170463733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf058cc00 con 0x55ddf0825ed0 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948db9a700 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 0x55ddf08371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948eb9c700 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf082f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f948e39b700 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0837ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9484006010 tx=0x7f9484005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70ded038a5299d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.160+0000 7f947d3ff700 1 -- 172.21.15.66:0/1170463733 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948402c070 con 0x55ddf0825ed0 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f947d3ff700 1 -- 172.21.15.66:0/1170463733 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9484021040 con 0x55ddf0825ed0 2023-12-11T19:35:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f947d3ff700 1 -- 172.21.15.66:0/1170463733 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484020040 con 0x55ddf0825ed0 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 -- 172.21.15.66:0/1170463733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 msgr2=0x55ddf0837ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0837ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9484006010 tx=0x7f9484005f80 comp rx=0 tx=0).stop 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 -- 172.21.15.66:0/1170463733 shutdown_connections 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0837ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0827720 0x55ddf082f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1170463733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0826830 0x55ddf08371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 -- 172.21.15.66:0/1170463733 >> 172.21.15.66:0/1170463733 conn(0x55ddf0755780 msgr2=0x55ddf0822460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 -- 172.21.15.66:0/1170463733 shutdown_connections 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.161+0000 7f94a4007b80 1 -- 172.21.15.66:0/1170463733 wait complete. 2023-12-11T19:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 -- start start 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 Processor -- start 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0834f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf0836460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf08369a0 con 0x55ddf0827720 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf0836b10 con 0x55ddf0825ed0 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf0836cd0 con 0x55ddf0826830 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.162+0000 7f94a4007b80 10 client.0 _ll_register_callbacks cb 0x55ddf065fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948eb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf0836460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0834f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0834f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:48220/0 (socket says 172.21.15.66:48220) 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 -- 172.21.15.66:0/1100782595 learned_addr learned my addr 172.21.15.66:0/1100782595 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948db9a700 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 -- 172.21.15.66:0/1100782595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 msgr2=0x55ddf08354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 -- 172.21.15.66:0/1100782595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 msgr2=0x55ddf0836460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf0836460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.163+0000 7f948e39b700 1 -- 172.21.15.66:0/1100782595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94881a6130 con 0x55ddf0825ed0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f948e39b700 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0834f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f948402a040 tx=0x7f9484005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7830244ce07fa8c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f9471ffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f9471ffb700 1 -- 172.21.15.66:0/1100782595 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948402c040 con 0x55ddf0825ed0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94840044b0 con 0x55ddf0825ed0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f9471ffb700 1 -- 172.21.15.66:0/1100782595 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9484004710 con 0x55ddf0825ed0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f9471ffb700 1 -- 172.21.15.66:0/1100782595 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94840048d0 con 0x55ddf0825ed0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f94a4007b80 10 client.25862 fetch_fsmap learned FSMap version 29 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ddf05f75d0 con 0x55ddf0825ed0 2023-12-11T19:35:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.164+0000 7f9471ffb700 1 -- 172.21.15.66:0/1100782595 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94840044b0 con 0x55ddf0825ed0 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f9471ffb700 1 -- 172.21.15.66:0/1100782595 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9484028730 con 0x55ddf0825ed0 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f94a4007b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 29 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f94a4007b80 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f94a4007b80 20 client.25862 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf0892e70 0x55ddf0895290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f94a4007b80 4 client.25862 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ddf0740f70 con 0x55ddf0892e70 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.165+0000 7f948db9a700 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf0892e70 0x55ddf0895290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.166+0000 7f948db9a700 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf0892e70 0x55ddf0895290 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10cca583940483c server_cookie=25043e585f21b6b4 in_seq=0 out_seq=0 2023-12-11T19:35:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.166+0000 7f9471ffb700 10 client.25862 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f9471ffb700 1 -- 172.21.15.66:0/1100782595 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ddf0740f70 con 0x55ddf0892e70 2023-12-11T19:35:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f9471ffb700 10 client.25862 handle_command_reply: tid=0 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 client.25862 shutdown 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 msgr2=0x55ddf0834f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0834f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f948402a040 tx=0x7f9484005020 comp rx=0 tx=0).stop 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 shutdown_connections 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ddf0892e70 0x55ddf0895290 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ddf0825ed0 0x55ddf0834f60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ddf0827720 0x55ddf0836460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 --2- 172.21.15.66:0/1100782595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddf0826830 0x55ddf08354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f9471ffb700 0 client.25862 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 >> 172.21.15.66:0/1100782595 conn(0x55ddf0755780 msgr2=0x55ddf082a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 shutdown_connections 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.175+0000 7f94a4007b80 1 -- 172.21.15.66:0/1100782595 wait complete. 2023-12-11T19:35:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94a4007b80 20 client.25862 trim_cache size 0 max 16384 2023-12-11T19:35:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:35:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:35:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:35:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f94740a1500 msgr2=0x7f94740a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f94740a1500 0x7f94740a39c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9480009cd0 tx=0x7f9480009c60 comp rx=0 tx=0).stop 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 msgr2=0x7f94881078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 0x7f94881078e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f947800d950 tx=0x7f947800dd10 comp rx=0 tx=0).stop 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 shutdown_connections 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f94740a1500 0x7f94740a39c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94881030c0 0x7f9488103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9488101530 0x7f94881073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 --2- 172.21.15.66:0/3400704465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102720 0x7f94881078e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 >> 172.21.15.66:0/3400704465 conn(0x7f94880fcaa0 msgr2=0x7f9488073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 shutdown_connections 2023-12-11T19:35:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:45.176+0000 7f94905ff700 1 -- 172.21.15.66:0/3400704465 wait complete. 2023-12-11T19:35:46.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:45 smithi167 ceph-mon[102750]: pgmap v425: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-12-11T19:35:46.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:45 smithi066 ceph-mon[97920]: pgmap v425: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-12-11T19:35:46.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:45 smithi070 ceph-mon[104044]: pgmap v425: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-12-11T19:35:48.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:47 smithi167 ceph-mon[102750]: pgmap v426: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 29 MiB/s wr, 19 op/s 2023-12-11T19:35:48.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:47 smithi066 ceph-mon[97920]: pgmap v426: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 29 MiB/s wr, 19 op/s 2023-12-11T19:35:48.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:47 smithi070 ceph-mon[104044]: pgmap v426: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 29 MiB/s wr, 19 op/s 2023-12-11T19:35:50.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:49 smithi167 ceph-mon[102750]: pgmap v427: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 18 MiB/s wr, 14 op/s 2023-12-11T19:35:50.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:49 smithi066 ceph-mon[97920]: pgmap v427: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 18 MiB/s wr, 14 op/s 2023-12-11T19:35:50.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:49 smithi070 ceph-mon[104044]: pgmap v427: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 18 MiB/s wr, 14 op/s 2023-12-11T19:35:50.319 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:50.319 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:50.320 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:50.320 INFO:tasks.workunit.client.0.smithi066.stdout:iozone test complete. 2023-12-11T19:35:50.323 INFO:tasks.workunit.client.0.smithi066.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2023-12-11T19:35:50.374 INFO:tasks.workunit.client.0.smithi066.stdout: Iozone: Performance Test of File I/O 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Version $Revision: 3.489 $ 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Compiled for 64 bit mode. 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Build: linux-AMD64 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2023-12-11T19:35:50.375 INFO:tasks.workunit.client.0.smithi066.stdout: Alexey Skidanov, Sudhir Kumar. 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Run began: Mon Dec 11 19:35:50 2023 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Include close in write timing 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Include fsync in write timing 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: File size set to 1048576 kB 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Record Size 1024 kB 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Output is in kBytes/sec 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Time Resolution = 0.000001 seconds. 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Processor cache size set to 1024 kBytes. 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Processor cache line size set to 32 bytes. 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: File stride size set to 17 * record size. 2023-12-11T19:35:50.376 INFO:tasks.workunit.client.0.smithi066.stdout: Throughput test with 1 process 2023-12-11T19:35:50.377 INFO:tasks.workunit.client.0.smithi066.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2023-12-11T19:35:52.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:51 smithi167 ceph-mon[102750]: pgmap v428: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 16 op/s 2023-12-11T19:35:52.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:51 smithi066 ceph-mon[97920]: pgmap v428: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 16 op/s 2023-12-11T19:35:52.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:51 smithi070 ceph-mon[104044]: pgmap v428: 97 pgs: 97 active+clean; 1.0 GiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 16 op/s 2023-12-11T19:35:53.238 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:35:53.239 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 initial writers = 370685.56 kB/sec 2023-12-11T19:35:53.239 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 initial writers = 370571.43 kB/sec 2023-12-11T19:35:53.239 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 370685.56 kB/sec 2023-12-11T19:35:53.239 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 370685.56 kB/sec 2023-12-11T19:35:53.239 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 370685.56 kB/sec 2023-12-11T19:35:53.239 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:35:54.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:53 smithi167 ceph-mon[102750]: pgmap v429: 97 pgs: 97 active+clean; 925 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.0 MiB/s wr, 15 op/s 2023-12-11T19:35:54.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:53 smithi066 ceph-mon[97920]: pgmap v429: 97 pgs: 97 active+clean; 925 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.0 MiB/s wr, 15 op/s 2023-12-11T19:35:54.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:53 smithi070 ceph-mon[104044]: pgmap v429: 97 pgs: 97 active+clean; 925 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.0 MiB/s wr, 15 op/s 2023-12-11T19:35:56.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:55 smithi167 ceph-mon[102750]: pgmap v430: 97 pgs: 97 active+clean; 933 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 51 MiB/s wr, 41 op/s 2023-12-11T19:35:56.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:55 smithi167 ceph-mon[102750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:56.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:55 smithi066 ceph-mon[97920]: pgmap v430: 97 pgs: 97 active+clean; 933 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 51 MiB/s wr, 41 op/s 2023-12-11T19:35:56.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:55 smithi066 ceph-mon[97920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:55 smithi070 ceph-mon[104044]: pgmap v430: 97 pgs: 97 active+clean; 933 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 51 MiB/s wr, 41 op/s 2023-12-11T19:35:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:55 smithi070 ceph-mon[104044]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:35:58.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:57 smithi167 ceph-mon[102750]: pgmap v431: 97 pgs: 97 active+clean; 981 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 83 MiB/s wr, 57 op/s 2023-12-11T19:35:58.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:57 smithi066 ceph-mon[97920]: pgmap v431: 97 pgs: 97 active+clean; 981 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 83 MiB/s wr, 57 op/s 2023-12-11T19:35:58.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:57 smithi070 ceph-mon[104044]: pgmap v431: 97 pgs: 97 active+clean; 981 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 83 MiB/s wr, 57 op/s 2023-12-11T19:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 -- 172.21.15.66:0/48974469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc1093f0 msgr2=0x7fc7cc10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/48974469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc1093f0 0x7fc7cc10b880 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400b0a0 tx=0x7fc7c400b3b0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 -- 172.21.15.66:0/48974469 shutdown_connections 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/48974469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc075610 0x7fc7cc073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/48974469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc1093f0 0x7fc7cc10b880 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/48974469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc074030 0x7fc7cc0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 -- 172.21.15.66:0/48974469 >> 172.21.15.66:0/48974469 conn(0x7fc7cc0fcaa0 msgr2=0x7fc7cc0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.328+0000 7fc7d33ff700 1 -- 172.21.15.66:0/48974469 shutdown_connections 2023-12-11T19:35:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.329+0000 7fc7d33ff700 1 -- 172.21.15.66:0/48974469 wait complete. 2023-12-11T19:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.330+0000 7fc7d33ff700 1 Processor -- start 2023-12-11T19:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.330+0000 7fc7d33ff700 1 -- start start 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d33ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc074030 0x7fc7cc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d33ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 0x7fc7cc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d33ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc1093f0 0x7fc7cc1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d33ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc1a11e0 con 0x7fc7cc075610 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d33ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc1a1320 con 0x7fc7cc074030 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d33ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc1a14b0 con 0x7fc7cc1093f0 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 0x7fc7cc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 0x7fc7cc199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60332/0 (socket says 172.21.15.66:60332) 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 -- 172.21.15.66:0/2257796950 learned_addr learned my addr 172.21.15.66:0/2257796950 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d199c700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc1093f0 0x7fc7cc1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d119b700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc074030 0x7fc7cc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 -- 172.21.15.66:0/2257796950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc1093f0 msgr2=0x7fc7cc1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc1093f0 0x7fc7cc1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 -- 172.21.15.66:0/2257796950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc074030 msgr2=0x7fc7cc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc074030 0x7fc7cc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d099a700 1 -- 172.21.15.66:0/2257796950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009d00 con 0x7fc7cc075610 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.331+0000 7fc7d119b700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc074030 0x7fc7cc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.332+0000 7fc7d099a700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 0x7fc7cc199bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e970 tx=0x7fc7bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.332+0000 7fc7c27fc700 1 -- 172.21.15.66:0/2257796950 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc009790 con 0x7fc7cc075610 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.332+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc1a1790 con 0x7fc7cc075610 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.332+0000 7fc7c27fc700 1 -- 172.21.15.66:0/2257796950 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7bc00eec0 con 0x7fc7cc075610 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.332+0000 7fc7c27fc700 1 -- 172.21.15.66:0/2257796950 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc003c10 con 0x7fc7cc075610 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.332+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc1a1cf0 con 0x7fc7cc075610 2023-12-11T19:35:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.335+0000 7fc7c27fc700 1 -- 172.21.15.66:0/2257796950 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc7bc003db0 con 0x7fc7cc075610 2023-12-11T19:35:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.337+0000 7fc7e6df3b80 1 Processor -- start 2023-12-11T19:35:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.337+0000 7fc7e6df3b80 1 -- start start 2023-12-11T19:35:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b4842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b485140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b3dfaf0 con 0x55ab0b483ed0 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b3dd410 con 0x55ab0b484830 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b3d8d00 con 0x55ab0b485720 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7c27fc700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc7b80a1500 0x7fc7b80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b4842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7c27fc700 1 -- 172.21.15.66:0/2257796950 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7bc014070 con 0x7fc7cc075610 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b485140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b4842f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60344/0 (socket says 172.21.15.66:60344) 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b485140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41288/0 (socket says 172.21.15.66:41288) 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d199c700 1 -- 172.21.15.66:0/3510035143 learned_addr learned my addr 172.21.15.66:0/3510035143 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d199c700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc7b80a1500 0x7fc7b80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d099a700 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 -- 172.21.15.66:0/3510035143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 msgr2=0x55ab0b485ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 -- 172.21.15.66:0/3510035143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 msgr2=0x55ab0b4842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b4842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d119b700 1 -- 172.21.15.66:0/3510035143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0b2555d0 con 0x55ab0b484830 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d099a700 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.338+0000 7fc7d199c700 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b4842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.339+0000 7fc7d119b700 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b485140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800bcd0 tx=0x7fc7c800f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54dfea5f1dad514 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.339+0000 7fc7d199c700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc7b80a1500 0x7fc7b80a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009cd0 tx=0x7fc7c400b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.339+0000 7fc7b77fe700 1 -- 172.21.15.66:0/3510035143 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800fbe0 con 0x55ab0b484830 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.339+0000 7fc7b77fe700 1 -- 172.21.15.66:0/3510035143 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7c800fd40 con 0x55ab0b484830 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.339+0000 7fc7b77fe700 1 -- 172.21.15.66:0/3510035143 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c801b650 con 0x55ab0b484830 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/3510035143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 msgr2=0x55ab0b485140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b485140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800bcd0 tx=0x7fc7c800f360 comp rx=0 tx=0).stop 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/3510035143 shutdown_connections 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b4842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/3510035143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b485140 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/3510035143 >> 172.21.15.66:0/3510035143 conn(0x55ab0b3b3780 msgr2=0x55ab0b480880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/3510035143 shutdown_connections 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.340+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/3510035143 wait complete. 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.341+0000 7fc7e6df3b80 1 -- start start 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.341+0000 7fc7e6df3b80 1 Processor -- start 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b48de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b48e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b4951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b392500 con 0x55ab0b484830 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b3921a0 con 0x55ab0b485720 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b2578c0 con 0x55ab0b483ed0 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7e6df3b80 10 client.0 _ll_register_callbacks cb 0x55ab0b2bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b48e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b48e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60360/0 (socket says 172.21.15.66:60360) 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 -- 172.21.15.66:0/113213996 learned_addr learned my addr 172.21.15.66:0/113213996 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d199c700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b4951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d119b700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b48de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 -- 172.21.15.66:0/113213996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 msgr2=0x55ab0b48de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b48de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 -- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 msgr2=0x55ab0b4951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b4951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.342+0000 7fc7d099a700 1 -- 172.21.15.66:0/113213996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc10c4a0 con 0x55ab0b484830 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7d099a700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b48e3c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0caa00 tx=0x7fc7bc09a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e028883b8e15f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/113213996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc099910 con 0x55ab0b484830 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/113213996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7bc09a220 con 0x55ab0b484830 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab0b477730 con 0x55ab0b484830 2023-12-11T19:35:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/113213996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7bc07a030 con 0x55ab0b484830 2023-12-11T19:35:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7e6df3b80 10 client.16146 fetch_fsmap learned FSMap version 30 2023-12-11T19:35:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fc7c800b900 con 0x55ab0b484830 2023-12-11T19:35:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/113213996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ab0b477730 con 0x55ab0b484830 2023-12-11T19:35:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/113213996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc7bc01fba0 con 0x55ab0b484830 2023-12-11T19:35:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7e6df3b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:35:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.343+0000 7fc7e6df3b80 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.344+0000 7fc7e6df3b80 20 client.16146 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.344+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b499d60 0x55ab0b49c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.344+0000 7fc7e6df3b80 4 client.16146 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.344+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab0b453ee0 con 0x55ab0b499d60 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.344+0000 7fc7d119b700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b499d60 0x55ab0b49c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.345+0000 7fc7d119b700 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b499d60 0x55ab0b49c180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33c54c40ed487e1 server_cookie=813f2410abb93ba in_seq=0 out_seq=0 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.345+0000 7fc7b4ff9700 10 client.16146 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/113213996 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab0b453ee0 con 0x55ab0b499d60 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7b4ff9700 10 client.16146 handle_command_reply: tid=0 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 client.16146 shutdown 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 msgr2=0x55ab0b48e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b48e3c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0caa00 tx=0x7fc7bc09a920 comp rx=0 tx=0).stop 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 shutdown_connections 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b499d60 0x55ab0b49c180 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b4951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7b4ff9700 0 client.16146 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b48de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/113213996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b48e3c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 >> 172.21.15.66:0/113213996 conn(0x55ab0b3b3780 msgr2=0x55ab0b47eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 shutdown_connections 2023-12-11T19:35:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/113213996 wait complete. 2023-12-11T19:35:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.346+0000 7fc7e6df3b80 20 client.16146 trim_cache size 0 max 16384 2023-12-11T19:35:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.362+0000 7fc7e6df3b80 1 Processor -- start 2023-12-11T19:35:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.362+0000 7fc7e6df3b80 1 -- start start 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.362+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b495ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.362+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b4951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b497180 con 0x55ab0b483ed0 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b484670 con 0x55ab0b484830 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b48c6b0 con 0x55ab0b485720 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b495ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b495ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60366/0 (socket says 172.21.15.66:60366) 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 -- 172.21.15.66:0/839527105 learned_addr learned my addr 172.21.15.66:0/839527105 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d199c700 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d099a700 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b4951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 -- 172.21.15.66:0/839527105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 msgr2=0x55ab0b485b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 -- 172.21.15.66:0/839527105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 msgr2=0x55ab0b4951b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b4951b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 -- 172.21.15.66:0/839527105 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0b201d80 con 0x55ab0b483ed0 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.363+0000 7fc7d119b700 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b495ad0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc7c801a000 tx=0x7fc7c800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b859da0cc490790c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7b7fff700 1 -- 172.21.15.66:0/839527105 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800feb0 con 0x55ab0b483ed0 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7b7fff700 1 -- 172.21.15.66:0/839527105 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7c802c070 con 0x55ab0b483ed0 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7b7fff700 1 -- 172.21.15.66:0/839527105 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8011910 con 0x55ab0b483ed0 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/839527105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 msgr2=0x55ab0b495ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b495ad0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc7c801a000 tx=0x7fc7c800b5c0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/839527105 shutdown_connections 2023-12-11T19:35:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b485720 0x55ab0b485b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b483ed0 0x55ab0b495ad0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/839527105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b484830 0x55ab0b4951b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/839527105 >> 172.21.15.66:0/839527105 conn(0x55ab0b3b3780 msgr2=0x55ab0b48d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/839527105 shutdown_connections 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.364+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/839527105 wait complete. 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 -- start start 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 Processor -- start 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b492f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b4934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b494480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b4949c0 con 0x55ab0b484830 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b494b30 con 0x55ab0b485720 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0b494cf0 con 0x55ab0b483ed0 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.365+0000 7fc7e6df3b80 10 client.0 _ll_register_callbacks cb 0x55ab0b2bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b4934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b492f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d199c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b494480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b492f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:45140/0 (socket says 172.21.15.66:45140) 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b4934c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60376/0 (socket says 172.21.15.66:60376) 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d199c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b494480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41312/0 (socket says 172.21.15.66:41312) 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 -- 172.21.15.66:0/2429569717 learned_addr learned my addr 172.21.15.66:0/2429569717 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 -- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 msgr2=0x55ab0b494480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b494480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 -- 172.21.15.66:0/2429569717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 msgr2=0x55ab0b4934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b4934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 -- 172.21.15.66:0/2429569717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab0b2555d0 con 0x55ab0b483ed0 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d099a700 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b4934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7d119b700 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b492f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005a80 tx=0x7fc7c800b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562c33840724cf25 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.366+0000 7fc7b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:35:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/2429569717 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8004040 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7c800fd40 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/2429569717 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7c80041a0 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/2429569717 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7c80043a0 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 10 client.25873 fetch_fsmap learned FSMap version 30 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ab0b1eac00 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/2429569717 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7c800fd40 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/2429569717 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc7c80039b0 con 0x55ab0b483ed0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 10 client.25873 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 20 client.25873 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b4f0fc0 0x55ab0b4f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.367+0000 7fc7e6df3b80 4 client.25873 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.368+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab0b39ef70 con 0x55ab0b4f0fc0 2023-12-11T19:35:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.368+0000 7fc7d099a700 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b4f0fc0 0x55ab0b4f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.368+0000 7fc7d099a700 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b4f0fc0 0x55ab0b4f33e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1cc56a4d57e702 server_cookie=a52513a835c95d26 in_seq=0 out_seq=0 2023-12-11T19:35:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.368+0000 7fc7b4ff9700 10 client.25873 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7b4ff9700 1 -- 172.21.15.66:0/2429569717 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab0b39ef70 con 0x55ab0b4f0fc0 2023-12-11T19:35:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7b4ff9700 10 client.25873 handle_command_reply: tid=0 2023-12-11T19:35:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 client.25873 shutdown 2023-12-11T19:35:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 msgr2=0x55ab0b492f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b492f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005a80 tx=0x7fc7c800b680 comp rx=0 tx=0).stop 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 shutdown_connections 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ab0b4f0fc0 0x55ab0b4f33e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ab0b485720 0x55ab0b494480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab0b483ed0 0x55ab0b492f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 --2- 172.21.15.66:0/2429569717 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ab0b484830 0x55ab0b4934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7b4ff9700 0 client.25873 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 >> 172.21.15.66:0/2429569717 conn(0x55ab0b3b3780 msgr2=0x55ab0b488940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 shutdown_connections 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 1 -- 172.21.15.66:0/2429569717 wait complete. 2023-12-11T19:35:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.378+0000 7fc7e6df3b80 20 client.25873 trim_cache size 0 max 16384 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc7b80a1500 msgr2=0x7fc7b80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc7b80a1500 0x7fc7b80a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009cd0 tx=0x7fc7c400b720 comp rx=0 tx=0).stop 2023-12-11T19:35:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 msgr2=0x7fc7cc199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 0x7fc7cc199bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e970 tx=0x7fc7bc00ec80 comp rx=0 tx=0).stop 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 shutdown_connections 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fc7b80a1500 0x7fc7b80a39c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc074030 0x7fc7cc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7cc1093f0 0x7fc7cc1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 --2- 172.21.15.66:0/2257796950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7cc075610 0x7fc7cc199bf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 >> 172.21.15.66:0/2257796950 conn(0x7fc7cc0fcaa0 msgr2=0x7fc7cc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 shutdown_connections 2023-12-11T19:35:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:35:58.379+0000 7fc7d33ff700 1 -- 172.21.15.66:0/2257796950 wait complete. 2023-12-11T19:35:59.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-11T19:35:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 -- 172.21.15.66:0/829771213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 msgr2=0x7f5ab4100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 --2- 172.21.15.66:0/829771213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab4100290 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009990 tx=0x7f5ab0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 -- 172.21.15.66:0/829771213 shutdown_connections 2023-12-11T19:35:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 --2- 172.21.15.66:0/829771213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab4101190 0x7f5ab4101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 --2- 172.21.15.66:0/829771213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab4100290 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 --2- 172.21.15.66:0/829771213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ab41007d0 0x7f5ab4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 -- 172.21.15.66:0/829771213 >> 172.21.15.66:0/829771213 conn(0x7f5ab40fb9f0 msgr2=0x7f5ab40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 -- 172.21.15.66:0/829771213 shutdown_connections 2023-12-11T19:35:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.336+0000 7f5abbbff700 1 -- 172.21.15.66:0/829771213 wait complete. 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.337+0000 7f5abbbff700 1 Processor -- start 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.337+0000 7f5abbbff700 1 -- start start 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5abbbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab41995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5abbbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab41007d0 0x7f5ab4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5abbbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ab4101190 0x7f5ab41a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5abbbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab41a1100 con 0x7f5ab40ffe70 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5abbbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab41a1240 con 0x7f5ab41007d0 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5abbbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab41a13d0 con 0x7f5ab4101190 2023-12-11T19:35:59.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab41995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab41995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60392/0 (socket says 172.21.15.66:60392) 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 -- 172.21.15.66:0/2653898736 learned_addr learned my addr 172.21.15.66:0/2653898736 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab919a700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab41007d0 0x7f5ab4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 -- 172.21.15.66:0/2653898736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ab4101190 msgr2=0x7f5ab41a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ab4101190 0x7f5ab41a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 -- 172.21.15.66:0/2653898736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab41007d0 msgr2=0x7f5ab4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab41007d0 0x7f5ab4199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab999b700 1 -- 172.21.15.66:0/2653898736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab00095f0 con 0x7f5ab40ffe70 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.338+0000 7f5ab919a700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab41007d0 0x7f5ab4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.339+0000 7f5ab999b700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab41995d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b2e0 tx=0x7f5ab0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.339+0000 7f5aaaffd700 1 -- 172.21.15.66:0/2653898736 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab001d070 con 0x7f5ab40ffe70 2023-12-11T19:35:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.339+0000 7f5aaaffd700 1 -- 172.21.15.66:0/2653898736 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab000be50 con 0x7f5ab40ffe70 2023-12-11T19:35:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.339+0000 7f5aaaffd700 1 -- 172.21.15.66:0/2653898736 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0021410 con 0x7f5ab40ffe70 2023-12-11T19:35:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.339+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab41a1650 con 0x7f5ab40ffe70 2023-12-11T19:35:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.339+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab41a1a70 con 0x7f5ab40ffe70 2023-12-11T19:35:59.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.342+0000 7f5aaaffd700 1 -- 172.21.15.66:0/2653898736 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5ab000f660 con 0x7f5ab40ffe70 2023-12-11T19:35:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.344+0000 7f5acf5cfb80 1 Processor -- start 2023-12-11T19:35:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.344+0000 7f5acf5cfb80 1 -- start start 2023-12-11T19:35:59.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.344+0000 7f5aaaffd700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5aa00a1500 0x7f5aa00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36eb2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5aaaffd700 1 -- 172.21.15.66:0/2653898736 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ab00ce220 con 0x7f5ab40ffe70 2023-12-11T19:35:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb2c20 0x55ca36eb30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36e0cb50 con 0x55ca36eb1d30 2023-12-11T19:35:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36e0a470 con 0x55ca36eb13d0 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36e05d60 con 0x55ca36eb2c20 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5aba19c700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5aa00a1500 0x7f5aa00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5ab919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36eb2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5ab999b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5ab999b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41346/0 (socket says 172.21.15.66:41346) 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5ab919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36eb2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60400/0 (socket says 172.21.15.66:60400) 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.345+0000 7f5ab999b700 1 -- 172.21.15.66:0/3002817802 learned_addr learned my addr 172.21.15.66:0/3002817802 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5ab999b700 1 -- 172.21.15.66:0/3002817802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb2c20 msgr2=0x55ca36eb30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5ab999b700 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb2c20 0x55ca36eb30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5ab999b700 1 -- 172.21.15.66:0/3002817802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 msgr2=0x55ca36eb2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5ab999b700 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36eb2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5ab999b700 1 -- 172.21.15.66:0/3002817802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca36c82630 con 0x55ca36eb13d0 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5aba19c700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5aa00a1500 0x7f5aa00a39c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008fd0 tx=0x7f5aac00d470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5ab999b700 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb17f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0097a90 tx=0x7f5ab0005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a2e661fcfe198b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5a9ffff700 1 -- 172.21.15.66:0/3002817802 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000f460 con 0x55ca36eb13d0 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5a9ffff700 1 -- 172.21.15.66:0/3002817802 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab009bc90 con 0x55ca36eb13d0 2023-12-11T19:35:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.346+0000 7f5a9ffff700 1 -- 172.21.15.66:0/3002817802 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab009aa80 con 0x55ca36eb13d0 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3002817802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 msgr2=0x55ca36eb17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb17f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0097a90 tx=0x7f5ab0005ac0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3002817802 shutdown_connections 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb17f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb2c20 0x55ca36eb30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3002817802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36eb2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3002817802 >> 172.21.15.66:0/3002817802 conn(0x55ca36de07e0 msgr2=0x55ca36eadd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3002817802 shutdown_connections 2023-12-11T19:35:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.347+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3002817802 wait complete. 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 -- start start 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 Processor -- start 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36ebae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ebb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36ec23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36dbf560 con 0x55ca36eb2c20 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36dbf200 con 0x55ca36eb13d0 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.348+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36c84920 con 0x55ca36eb1d30 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5acf5cfb80 10 client.0 _ll_register_callbacks cb 0x55ca36ceaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36ec23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5ab999b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36ebae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5ab999b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36ebae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41348/0 (socket says 172.21.15.66:41348) 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5ab999b700 1 -- 172.21.15.66:0/3306321696 learned_addr learned my addr 172.21.15.66:0/3306321696 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:59.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5ab919a700 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ebb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 -- 172.21.15.66:0/3306321696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 msgr2=0x55ca36ebb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ebb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 -- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 msgr2=0x55ca36ebae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36ebae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 -- 172.21.15.66:0/3306321696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab41a1a70 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5aba19c700 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36ec23f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5aac004aa0 tx=0x7f5aac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65950da6a313d612 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.349+0000 7f5a9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/3306321696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00dee0 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca36ea4b40 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/3306321696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5aac0167e0 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/3306321696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac0169a0 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5acf5cfb80 10 client.16164 fetch_fsmap learned FSMap version 30 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ca36c2ede0 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/3306321696 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ca36ea4b40 con 0x55ca36eb2c20 2023-12-11T19:35:59.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/3306321696 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5aac022ac0 con 0x55ca36eb2c20 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5acf5cfb80 10 client.16164 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5acf5cfb80 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.350+0000 7f5acf5cfb80 20 client.16164 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.351+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36ec6d70 0x55ca36ec9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.351+0000 7f5acf5cfb80 4 client.16164 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.351+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca36e80f40 con 0x55ca36ec6d70 2023-12-11T19:35:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.351+0000 7f5ab999b700 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36ec6d70 0x55ca36ec9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.352+0000 7f5a9d7fa700 10 client.16164 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.352+0000 7f5ab999b700 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36ec6d70 0x55ca36ec9190 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f527ee484c7dc810 server_cookie=b262058c392f4448 in_seq=0 out_seq=0 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/3306321696 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca36e80f40 con 0x55ca36ec6d70 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5a9d7fa700 10 client.16164 handle_command_reply: tid=0 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 client.16164 shutdown 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 msgr2=0x55ca36ec23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36ec23f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5aac004aa0 tx=0x7f5aac004ad0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 shutdown_connections 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36ec6d70 0x55ca36ec9190 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36ebae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36ec23f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/3306321696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ebb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5a9d7fa700 0 client.16164 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 >> 172.21.15.66:0/3306321696 conn(0x55ca36de07e0 msgr2=0x55ca36eaca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 shutdown_connections 2023-12-11T19:35:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.353+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/3306321696 wait complete. 2023-12-11T19:35:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.354+0000 7f5acf5cfb80 20 client.16164 trim_cache size 0 max 16384 2023-12-11T19:35:59.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 Processor -- start 2023-12-11T19:35:59.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 -- start start 2023-12-11T19:35:59.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ec23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36eb5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36ec3570 con 0x55ca36eb2c20 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36eba9e0 con 0x55ca36eb13d0 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.372+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36eb2480 con 0x55ca36eb1d30 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36eb5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36eb5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60406/0 (socket says 172.21.15.66:60406) 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 -- 172.21.15.66:0/112590153 learned_addr learned my addr 172.21.15.66:0/112590153 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5ab999b700 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5ab919a700 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ec23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 -- 172.21.15.66:0/112590153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 msgr2=0x55ca36ec23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ec23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 -- 172.21.15.66:0/112590153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 msgr2=0x55ca36eb3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 -- 172.21.15.66:0/112590153 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca36c17c60 con 0x55ca36eb2c20 2023-12-11T19:35:59.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aba19c700 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36eb5af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0163a0 tx=0x7f5aac02c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae48a19d0bb4ce58 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aa8bff700 1 -- 172.21.15.66:0/112590153 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00dec0 con 0x55ca36eb2c20 2023-12-11T19:35:59.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.373+0000 7f5aa8bff700 1 -- 172.21.15.66:0/112590153 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aac008d00 con 0x55ca36eb2c20 2023-12-11T19:35:59.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5aa8bff700 1 -- 172.21.15.66:0/112590153 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0223f0 con 0x55ca36eb2c20 2023-12-11T19:35:59.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/112590153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 msgr2=0x55ca36eb5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36eb5af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0163a0 tx=0x7f5aac02c410 comp rx=0 tx=0).stop 2023-12-11T19:35:59.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/112590153 shutdown_connections 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb13d0 0x55ca36eb3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb2c20 0x55ca36eb5af0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/112590153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb1d30 0x55ca36ec23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/112590153 >> 172.21.15.66:0/112590153 conn(0x55ca36de07e0 msgr2=0x55ca36ecdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/112590153 shutdown_connections 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.374+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/112590153 wait complete. 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 -- start start 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 Processor -- start 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb13d0 0x55ca36ec1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36ec2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb2c20 0x55ca36ebe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36ebeb10 con 0x55ca36eb1d30 2023-12-11T19:35:59.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36ebec80 con 0x55ca36eb2c20 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca36ebee40 con 0x55ca36eb13d0 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5acf5cfb80 10 client.0 _ll_register_callbacks cb 0x55ca36ceaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.375+0000 7f5ab919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36ec2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36ec2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60420/0 (socket says 172.21.15.66:60420) 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 -- 172.21.15.66:0/1820313176 learned_addr learned my addr 172.21.15.66:0/1820313176 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab999b700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb13d0 0x55ca36ec1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 -- 172.21.15.66:0/1820313176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb13d0 msgr2=0x55ca36ec1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5aba19c700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb2c20 0x55ca36ebe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb13d0 0x55ca36ec1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 -- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb2c20 msgr2=0x55ca36ebe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb2c20 0x55ca36ebe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 -- 172.21.15.66:0/1820313176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca36c2ede0 con 0x55ca36eb1d30 2023-12-11T19:35:59.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5aba19c700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb2c20 0x55ca36ebe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5ab919a700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36ec2070 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4012840 tx=0x7f5aa4012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48edca68c386626 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5a9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/1820313176 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4011910 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/1820313176 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5aa4011a70 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.376+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ab009bc90 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/1820313176 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa4011c30 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 10 client.16170 fetch_fsmap learned FSMap version 30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ca36c55250 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/1820313176 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab009bc90 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/1820313176 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5aa40283e0 con 0x55ca36eb1d30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 10 client.16170 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 20 client.16170 populate_metadata read hostname 'smithi066' 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36f1e800 0x55ca36f20c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 4 client.16170 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.377+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca36dcbfd0 con 0x55ca36f1e800 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.378+0000 7f5ab999b700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36f1e800 0x55ca36f20c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.378+0000 7f5ab999b700 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36f1e800 0x55ca36f20c20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc648d4a8992d76 server_cookie=c1e8caa9ad02a8e3 in_seq=0 out_seq=0 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.378+0000 7f5a9d7fa700 10 client.16170 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.378+0000 7f5a9d7fa700 1 -- 172.21.15.66:0/1820313176 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ca36dcbfd0 con 0x55ca36f1e800 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.378+0000 7f5a9d7fa700 10 client.16170 handle_command_reply: tid=0 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.378+0000 7f5acf5cfb80 1 client.16170 shutdown 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 msgr2=0x55ca36ec2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36ec2070 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4012840 tx=0x7f5aa4012b50 comp rx=0 tx=0).stop 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 shutdown_connections 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x55ca36f1e800 0x55ca36f20c20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ca36eb2c20 0x55ca36ebe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca36eb13d0 0x55ca36ec1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5a9d7fa700 0 client.16170 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 --2- 172.21.15.66:0/1820313176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ca36eb1d30 0x55ca36ec2070 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 >> 172.21.15.66:0/1820313176 conn(0x55ca36de07e0 msgr2=0x55ca36eb5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 shutdown_connections 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 1 -- 172.21.15.66:0/1820313176 wait complete. 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5acf5cfb80 20 client.16170 trim_cache size 0 max 16384 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-12-11T19:35:59.384 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:35:59.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5aa00a1500 msgr2=0x7f5aa00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5abbbff700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5aa00a1500 0x7f5aa00a39c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008fd0 tx=0x7f5aac00d470 comp rx=0 tx=0).stop 2023-12-11T19:35:59.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 msgr2=0x7f5ab41995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.379+0000 7f5abbbff700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab41995d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b2e0 tx=0x7f5ab0005e80 comp rx=0 tx=0).stop 2023-12-11T19:35:59.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 shutdown_connections 2023-12-11T19:35:59.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5aa00a1500 0x7f5aa00a39c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ab4101190 0x7f5ab41a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ab40ffe70 0x7f5ab41995d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 --2- 172.21.15.66:0/2653898736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ab41007d0 0x7f5ab4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 >> 172.21.15.66:0/2653898736 conn(0x7f5ab40fb9f0 msgr2=0x7f5ab40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 shutdown_connections 2023-12-11T19:35:59.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.380+0000 7f5abbbff700 1 -- 172.21.15.66:0/2653898736 wait complete. 2023-12-11T19:35:59.398 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-11T19:35:59.398 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T19:35:59.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 -- 172.21.15.66:0/1578136226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 msgr2=0x7f1a5c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 --2- 172.21.15.66:0/1578136226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c100b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009990 tx=0x7f1a44009ca0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 -- 172.21.15.66:0/1578136226 shutdown_connections 2023-12-11T19:35:59.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 --2- 172.21.15.66:0/1578136226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5c0ffd70 0x7f1a5c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 --2- 172.21.15.66:0/1578136226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c101090 0x7f1a5c101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 --2- 172.21.15.66:0/1578136226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c100b50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 -- 172.21.15.66:0/1578136226 >> 172.21.15.66:0/1578136226 conn(0x7f1a5c0fb8f0 msgr2=0x7f1a5c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 -- 172.21.15.66:0/1578136226 shutdown_connections 2023-12-11T19:35:59.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.606+0000 7f1a61fff700 1 -- 172.21.15.66:0/1578136226 wait complete. 2023-12-11T19:35:59.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.607+0000 7f1a61fff700 1 Processor -- start 2023-12-11T19:35:59.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 -- start start 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c0ffd70 0x7f1a5c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5c101090 0x7f1a5c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1a1070 con 0x7f1a5c1006d0 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1a11b0 con 0x7f1a5c101090 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.608+0000 7f1a61fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1a1340 con 0x7f1a5c0ffd70 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60440/0 (socket says 172.21.15.66:60440) 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 -- 172.21.15.66:0/2009849837 learned_addr learned my addr 172.21.15.66:0/2009849837 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a5b7fe700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c0ffd70 0x7f1a5c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a5bfff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5c101090 0x7f1a5c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 -- 172.21.15.66:0/2009849837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c0ffd70 msgr2=0x7f1a5c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c0ffd70 0x7f1a5c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 -- 172.21.15.66:0/2009849837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5c101090 msgr2=0x7f1a5c1a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5c101090 0x7f1a5c1a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 -- 172.21.15.66:0/2009849837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440095f0 con 0x7f1a5c1006d0 2023-12-11T19:35:59.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a53fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c199a80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a44009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.609+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4401d070 con 0x7f1a5c1006d0 2023-12-11T19:35:59.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.610+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a4400bed0 con 0x7f1a5c1006d0 2023-12-11T19:35:59.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.610+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c1a15c0 con 0x7f1a5c1006d0 2023-12-11T19:35:59.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.610+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44021410 con 0x7f1a5c1006d0 2023-12-11T19:35:59.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.610+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c1a1a40 con 0x7f1a5c1006d0 2023-12-11T19:35:59.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.611+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c10ae70 con 0x7f1a5c1006d0 2023-12-11T19:35:59.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.613+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1a4400f460 con 0x7f1a5c1006d0 2023-12-11T19:35:59.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.616+0000 7f1a597fa700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1a480a1500 0x7f1a480a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:35:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.616+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1a440ce470 con 0x7f1a5c1006d0 2023-12-11T19:35:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.616+0000 7f1a5b7fe700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1a480a1500 0x7f1a480a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:35:59.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.617+0000 7f1a5b7fe700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1a480a1500 0x7f1a480a39c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009bf0 tx=0x7f1a4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:35:59.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.618+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1a44097b20 con 0x7f1a5c1006d0 2023-12-11T19:35:59.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:35:59 smithi066 ceph-mon[97920]: pgmap v432: 97 pgs: 97 active+clean; 1.0 GiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 109 MiB/s wr, 61 op/s 2023-12-11T19:35:59.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.879+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a5c068230 con 0x7f1a5c1006d0 2023-12-11T19:35:59.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.880+0000 7f1a597fa700 1 -- 172.21.15.66:0/2009849837 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11364 (secure 0 0 0) 0x7f1a4409b140 con 0x7f1a5c1006d0 2023-12-11T19:35:59.880 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:35:59.880 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1528570111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1528570111},{"type":"v1","addr":"172.21.15.66:6835","nonce":1528570111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/4055829654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":4055829654},{"type":"v1","addr":"172.21.15.66:6837","nonce":4055829654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T19:31:51.764249+0000","modified":"2023-12-11T19:35:53.864710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24547,"mds_2":24514,"mds_3":24494,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14673":{"gid":14673,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2148751938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2148751938},{"type":"v1","addr":"172.21.15.66:6839","nonce":2148751938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14685":{"gid":14685,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6841/1874182059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":1874182059},{"type":"v1","addr":"172.21.15.66:6841","nonce":1874182059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1537942709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1537942709},{"type":"v1","addr":"172.21.15.167:6833","nonce":1537942709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":66,"addr":"172.21.15.70:6833/1581467356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1581467356},{"type":"v1","addr":"172.21.15.70:6833","nonce":1581467356}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/3670150002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3670150002},{"type":"v1","addr":"172.21.15.70:6835","nonce":3670150002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.167:6835/1867143564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1867143564},{"type":"v1","addr":"172.21.15.167:6835","nonce":1867143564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2775450491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2775450491},{"type":"v1","addr":"172.21.15.167:6837","nonce":2775450491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24533":{"gid":24533,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6837/280227891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":280227891},{"type":"v1","addr":"172.21.15.70:6837","nonce":280227891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.167:6839/3906090427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3906090427},{"type":"v1","addr":"172.21.15.167:6839","nonce":3906090427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6839/725803001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6838","nonce":725803001},{"type":"v1","addr":"172.21.15.70:6839","nonce":725803001}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T19:35:59.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1a480a1500 msgr2=0x7f1a480a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1a480a1500 0x7f1a480a39c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009bf0 tx=0x7f1a4c009260 comp rx=0 tx=0).stop 2023-12-11T19:35:59.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 msgr2=0x7f1a5c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:35:59.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c199a80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1a44006010 tx=0x7f1a44009510 comp rx=0 tx=0).stop 2023-12-11T19:35:59.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 shutdown_connections 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f1a480a1500 0x7f1a480a39c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a5c101090 0x7f1a5c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c0ffd70 0x7f1a5c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 --2- 172.21.15.66:0/2009849837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a5c1006d0 0x7f1a5c199a80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 >> 172.21.15.66:0/2009849837 conn(0x7f1a5c0fb8f0 msgr2=0x7f1a5c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 shutdown_connections 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:35:59.882+0000 7f1a61fff700 1 -- 172.21.15.66:0/2009849837 wait complete. 2023-12-11T19:35:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 30 2023-12-11T19:35:59.901 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-11T19:36:00.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:35:59 smithi167 ceph-mon[102750]: pgmap v432: 97 pgs: 97 active+clean; 1.0 GiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 109 MiB/s wr, 61 op/s 2023-12-11T19:36:00.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:35:59 smithi070 ceph-mon[104044]: pgmap v432: 97 pgs: 97 active+clean; 1.0 GiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 109 MiB/s wr, 61 op/s 2023-12-11T19:36:00.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 -- 172.21.15.66:0/2008858900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701015a0 msgr2=0x7f38701019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 --2- 172.21.15.66:0/2008858900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701015a0 0x7f38701019c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f386c0099c0 tx=0x7f386c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 -- 172.21.15.66:0/2008858900 shutdown_connections 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 --2- 172.21.15.66:0/2008858900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3870103130 0x7f38701099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 --2- 172.21.15.66:0/2008858900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38701015a0 0x7f38701019c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 --2- 172.21.15.66:0/2008858900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3870102790 0x7f3870102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.111+0000 7f38775ff700 1 -- 172.21.15.66:0/2008858900 >> 172.21.15.66:0/2008858900 conn(0x7f38700fcb10 msgr2=0x7f38700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.112+0000 7f38775ff700 1 -- 172.21.15.66:0/2008858900 shutdown_connections 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.112+0000 7f38775ff700 1 -- 172.21.15.66:0/2008858900 wait complete. 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 Processor -- start 2023-12-11T19:36:00.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 -- start start 2023-12-11T19:36:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38701015a0 0x7f3870199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 0x7f3870199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3870103130 0x7f38701a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38701a1240 con 0x7f3870102790 2023-12-11T19:36:00.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38701a1380 con 0x7f38701015a0 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f38775ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38701a1510 con 0x7f3870103130 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 0x7f3870199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 0x7f3870199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60456/0 (socket says 172.21.15.66:60456) 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.113+0000 7f3874b9a700 1 -- 172.21.15.66:0/2339947110 learned_addr learned my addr 172.21.15.66:0/2339947110 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f387539b700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38701015a0 0x7f3870199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f3874b9a700 1 -- 172.21.15.66:0/2339947110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3870103130 msgr2=0x7f38701a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f3874b9a700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3870103130 0x7f38701a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f3874b9a700 1 -- 172.21.15.66:0/2339947110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38701015a0 msgr2=0x7f3870199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f3874b9a700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38701015a0 0x7f3870199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f3874b9a700 1 -- 172.21.15.66:0/2339947110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0095f0 con 0x7f3870102790 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f3874b9a700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 0x7f3870199ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f386000e9a0 tx=0x7f386000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f38667fc700 1 -- 172.21.15.66:0/2339947110 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860009790 con 0x7f3870102790 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f38667fc700 1 -- 172.21.15.66:0/2339947110 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3860010430 con 0x7f3870102790 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38701a17f0 con 0x7f3870102790 2023-12-11T19:36:00.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f38667fc700 1 -- 172.21.15.66:0/2339947110 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860018680 con 0x7f3870102790 2023-12-11T19:36:00.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.114+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38701a1d00 con 0x7f3870102790 2023-12-11T19:36:00.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.118+0000 7f38667fc700 1 -- 172.21.15.66:0/2339947110 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f386000cb50 con 0x7f3870102790 2023-12-11T19:36:00.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 Processor -- start 2023-12-11T19:36:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 -- start start 2023-12-11T19:36:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f38667fc700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f385c0a1480 0x7f385c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82ca3d0 0x55f9d82ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f38667fc700 1 -- 172.21.15.66:0/2339947110 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3860014070 con 0x7f3870102790 2023-12-11T19:36:00.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82cbc20 0x55f9d82cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d8225b50 con 0x55f9d82cad30 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d8223470 con 0x55f9d82ca3d0 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.119+0000 7f388af5db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d821ed60 con 0x55f9d82cbc20 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3875b9c700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f385c0a1480 0x7f385c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60464/0 (socket says 172.21.15.66:60464) 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 -- 172.21.15.66:0/1462182638 learned_addr learned my addr 172.21.15.66:0/1462182638 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f387539b700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82ca3d0 0x55f9d82ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3875b9c700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82cbc20 0x55f9d82cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 -- 172.21.15.66:0/1462182638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82cbc20 msgr2=0x55f9d82cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82cbc20 0x55f9d82cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 -- 172.21.15.66:0/1462182638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82ca3d0 msgr2=0x55f9d82ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82ca3d0 0x55f9d82ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 -- 172.21.15.66:0/1462182638 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d809b630 con 0x55f9d82cad30 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f387539b700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82ca3d0 0x55f9d82ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3875b9c700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82cbc20 0x55f9d82cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3875b9c700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f385c0a1480 0x7f385c0a3940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3868009fd0 tx=0x7f3868009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.120+0000 7f3874b9a700 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82cb640 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f38600a2130 tx=0x7f3860099cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f1c8f0a7bbc9d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.121+0000 7f385b7fe700 1 -- 172.21.15.66:0/1462182638 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386009a9d0 con 0x55f9d82cad30 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.121+0000 7f385b7fe700 1 -- 172.21.15.66:0/1462182638 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386009a2d0 con 0x55f9d82cad30 2023-12-11T19:36:00.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.121+0000 7f385b7fe700 1 -- 172.21.15.66:0/1462182638 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860098670 con 0x55f9d82cad30 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.121+0000 7f388af5db80 1 -- 172.21.15.66:0/1462182638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 msgr2=0x55f9d82cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.121+0000 7f388af5db80 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82cb640 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f38600a2130 tx=0x7f3860099cb0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 -- 172.21.15.66:0/1462182638 shutdown_connections 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82ca3d0 0x55f9d82ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82cbc20 0x55f9d82cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 --2- 172.21.15.66:0/1462182638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82cb640 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 -- 172.21.15.66:0/1462182638 >> 172.21.15.66:0/1462182638 conn(0x55f9d81f97e0 msgr2=0x55f9d82c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 -- 172.21.15.66:0/1462182638 shutdown_connections 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 -- 172.21.15.66:0/1462182638 wait complete. 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 -- start start 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.122+0000 7f388af5db80 1 Processor -- start 2023-12-11T19:36:00.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d81d8560 con 0x55f9d82cad30 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d81d8200 con 0x55f9d82cbc20 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d809d920 con 0x55f9d82ca3d0 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f388af5db80 10 client.0 _ll_register_callbacks cb 0x55f9d8103a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60470/0 (socket says 172.21.15.66:60470) 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f3874b9a700 1 -- 172.21.15.66:0/1156137148 learned_addr learned my addr 172.21.15.66:0/1156137148 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f387539b700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.123+0000 7f3875b9c700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3874b9a700 1 -- 172.21.15.66:0/1156137148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 msgr2=0x55f9d82d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3874b9a700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3874b9a700 1 -- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 msgr2=0x55f9d82db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3874b9a700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3874b9a700 1 -- 172.21.15.66:0/1156137148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38701a1d00 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3874b9a700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d4770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3860096d40 tx=0x7f3860010a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2f7a13f134f820 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3858ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3858ff9700 1 -- 172.21.15.66:0/1156137148 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a2660 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9d82bdb40 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3858ff9700 1 -- 172.21.15.66:0/1156137148 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38600105a0 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f3858ff9700 1 -- 172.21.15.66:0/1156137148 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38600a21d0 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f388af5db80 10 client.16188 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.124+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f9d8047de0 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f3858ff9700 1 -- 172.21.15.66:0/1156137148 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f9d82bdb40 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f3858ff9700 1 -- 172.21.15.66:0/1156137148 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3860099380 con 0x55f9d82cad30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f388af5db80 10 client.16188 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f388af5db80 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f388af5db80 20 client.16188 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f388af5db80 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d82dfd30 0x55f9d82e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f388af5db80 4 client.16188 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d8299f40 con 0x55f9d82dfd30 2023-12-11T19:36:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.125+0000 7f387539b700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d82dfd30 0x55f9d82e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.126+0000 7f387539b700 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d82dfd30 0x55f9d82e2150 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9fdac6e7a908a694 server_cookie=63fac10e3747a6d8 in_seq=0 out_seq=0 2023-12-11T19:36:00.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.126+0000 7f3858ff9700 10 client.16188 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f3858ff9700 1 -- 172.21.15.66:0/1156137148 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9d8299f40 con 0x55f9d82dfd30 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f3858ff9700 10 client.16188 handle_command_reply: tid=0 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 client.16188 shutdown 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 msgr2=0x55f9d82d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d4770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3860096d40 tx=0x7f3860010a50 comp rx=0 tx=0).stop 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 shutdown_connections 2023-12-11T19:36:00.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d82dfd30 0x55f9d82e2150 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f3858ff9700 0 client.16188 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 --2- 172.21.15.66:0/1156137148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d4770 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 >> 172.21.15.66:0/1156137148 conn(0x55f9d81f97e0 msgr2=0x55f9d82c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 shutdown_connections 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.127+0000 7f388af5db80 1 -- 172.21.15.66:0/1156137148 wait complete. 2023-12-11T19:36:00.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.128+0000 7f388af5db80 20 client.16188 trim_cache size 0 max 16384 2023-12-11T19:36:00.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 Processor -- start 2023-12-11T19:36:00.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 -- start start 2023-12-11T19:36:00.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d82bdb20 con 0x55f9d82cad30 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d82c9f70 con 0x55f9d82cbc20 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.141+0000 7f388af5db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d82cb250 con 0x55f9d82ca3d0 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82db770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60476/0 (socket says 172.21.15.66:60476) 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 -- 172.21.15.66:0/3136574873 learned_addr learned my addr 172.21.15.66:0/3136574873 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3875b9c700 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 -- 172.21.15.66:0/3136574873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 msgr2=0x55f9d82dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 -- 172.21.15.66:0/3136574873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 msgr2=0x55f9d82d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 -- 172.21.15.66:0/3136574873 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d8030c60 con 0x55f9d82cad30 2023-12-11T19:36:00.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f3874b9a700 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82db770 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f386009a210 tx=0x7f3860098880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963ce9f41955e22e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f385bfff700 1 -- 172.21.15.66:0/3136574873 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860098480 con 0x55f9d82cad30 2023-12-11T19:36:00.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.142+0000 7f385bfff700 1 -- 172.21.15.66:0/3136574873 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38600a2660 con 0x55f9d82cad30 2023-12-11T19:36:00.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f385bfff700 1 -- 172.21.15.66:0/3136574873 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a2430 con 0x55f9d82cad30 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 -- 172.21.15.66:0/3136574873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 msgr2=0x55f9d82db770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82db770 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f386009a210 tx=0x7f3860098880 comp rx=0 tx=0).stop 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 -- 172.21.15.66:0/3136574873 shutdown_connections 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 --2- 172.21.15.66:0/3136574873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82db770 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 -- 172.21.15.66:0/3136574873 >> 172.21.15.66:0/3136574873 conn(0x55f9d81f97e0 msgr2=0x55f9d82e6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 -- 172.21.15.66:0/3136574873 shutdown_connections 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.143+0000 7f388af5db80 1 -- 172.21.15.66:0/3136574873 wait complete. 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 -- start start 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 Processor -- start 2023-12-11T19:36:00.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d82da8b0 con 0x55f9d82cad30 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d82daa20 con 0x55f9d82cbc20 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d82dabe0 con 0x55f9d82ca3d0 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f388af5db80 10 client.0 _ll_register_callbacks cb 0x55f9d8103a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.144+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:45280/0 (socket says 172.21.15.66:45280) 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 -- 172.21.15.66:0/4275209990 learned_addr learned my addr 172.21.15.66:0/4275209990 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f3874b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60490/0 (socket says 172.21.15.66:60490) 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f3875b9c700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 -- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 msgr2=0x55f9d82da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 -- 172.21.15.66:0/4275209990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 msgr2=0x55f9d82d93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 -- 172.21.15.66:0/4275209990 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d8047de0 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f3874b9a700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f387539b700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d8e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ed55df9d8c4dc04 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.145+0000 7f3858ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f3858ff9700 1 -- 172.21.15.66:0/4275209990 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c01d030 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f386009a2d0 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f3858ff9700 1 -- 172.21.15.66:0/4275209990 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f386c028e40 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f3858ff9700 1 -- 172.21.15.66:0/4275209990 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f386c00bd20 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 10 client.25900 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f9d806e250 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f3858ff9700 1 -- 172.21.15.66:0/4275209990 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386009a2d0 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f3858ff9700 1 -- 172.21.15.66:0/4275209990 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f386c0280b0 con 0x55f9d82ca3d0 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 10 client.25900 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 10 client.25900 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-12-11T19:36:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 20 client.25900 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.146+0000 7f388af5db80 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d8336b20 0x55f9d8338f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.147+0000 7f388af5db80 4 client.25900 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:36:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.147+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 --> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f9d81e4fd0 con 0x55f9d8336b20 2023-12-11T19:36:00.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.147+0000 7f3874b9a700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d8336b20 0x55f9d8338f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f3874b9a700 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d8336b20 0x55f9d8338f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed85eb28b00f78fb server_cookie=f79bc54448fce36c in_seq=0 out_seq=0 2023-12-11T19:36:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f3858ff9700 10 client.25900 ms_handle_connect on v2:172.21.15.70:6832/1581467356 2023-12-11T19:36:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f3858ff9700 1 -- 172.21.15.66:0/4275209990 <== mds.4 v2:172.21.15.70:6832/1581467356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+202 (crc 0 0 0) 0x55f9d81e4fd0 con 0x55f9d8336b20 2023-12-11T19:36:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f3858ff9700 10 client.25900 handle_command_reply: tid=0 2023-12-11T19:36:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 client.25900 shutdown 2023-12-11T19:36:00.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 msgr2=0x55f9d82d8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d8e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c005e80 comp rx=0 tx=0).stop 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 shutdown_connections 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:6832/1581467356,v1:172.21.15.70:6833/1581467356] conn(0x55f9d8336b20 0x55f9d8338f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f9d82cbc20 0x55f9d82da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9d82ca3d0 0x55f9d82d8e70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f3858ff9700 0 client.25900 ms_handle_reset on v2:172.21.15.70:6832/1581467356 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 --2- 172.21.15.66:0/4275209990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f9d82cad30 0x55f9d82d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 >> 172.21.15.66:0/4275209990 conn(0x55f9d81f97e0 msgr2=0x55f9d82d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 shutdown_connections 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 1 -- 172.21.15.66:0/4275209990 wait complete. 2023-12-11T19:36:00.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.148+0000 7f388af5db80 20 client.25900 trim_cache size 0 max 16384 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 1807753778, 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627783, 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106/client.0/tmp/f1" 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:36:00.153 INFO:teuthology.orchestra.run.smithi066.stdout:] 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f385c0a1480 msgr2=0x7f385c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f385c0a1480 0x7f385c0a3940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3868009fd0 tx=0x7f3868009da0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 msgr2=0x7f3870199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 0x7f3870199ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f386000e9a0 tx=0x7f386000ed60 comp rx=0 tx=0).stop 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 shutdown_connections 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f385c0a1480 0x7f385c0a3940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38701015a0 0x7f3870199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3870103130 0x7f38701a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 --2- 172.21.15.66:0/2339947110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3870102790 0x7f3870199ca0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 >> 172.21.15.66:0/2339947110 conn(0x7f38700fcb10 msgr2=0x7f3870107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 shutdown_connections 2023-12-11T19:36:00.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.149+0000 7f38775ff700 1 -- 172.21.15.66:0/2339947110 wait complete. 2023-12-11T19:36:00.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-11T19:36:00.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 -- 172.21.15.66:0/225682038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0102790 msgr2=0x7f24d0102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 --2- 172.21.15.66:0/225682038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0102790 0x7f24d0102bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f24b80099c0 tx=0x7f24b8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 -- 172.21.15.66:0/225682038 shutdown_connections 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 --2- 172.21.15.66:0/225682038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d0103130 0x7f24d01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 --2- 172.21.15.66:0/225682038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24d01015a0 0x7f24d01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 --2- 172.21.15.66:0/225682038 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0102790 0x7f24d0102bf0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 -- 172.21.15.66:0/225682038 >> 172.21.15.66:0/225682038 conn(0x7f24d00fcb10 msgr2=0x7f24d00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 -- 172.21.15.66:0/225682038 shutdown_connections 2023-12-11T19:36:00.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.380+0000 7f24d51ff700 1 -- 172.21.15.66:0/225682038 wait complete. 2023-12-11T19:36:00.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.381+0000 7f24d51ff700 1 Processor -- start 2023-12-11T19:36:00.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.381+0000 7f24d51ff700 1 -- start start 2023-12-11T19:36:00.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24d51ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 0x7f24d01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24d51ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24d0102790 0x7f24d0199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24d51ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 0x7f24d01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24d51ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a11a0 con 0x7f24d0103130 2023-12-11T19:36:00.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24d51ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a12e0 con 0x7f24d01015a0 2023-12-11T19:36:00.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24d51ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a1470 con 0x7f24d0102790 2023-12-11T19:36:00.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24ced9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 0x7f24d01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24cf59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 0x7f24d01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24ced9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 0x7f24d01996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41426/0 (socket says 172.21.15.66:41426) 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24d0102790 0x7f24d0199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24ced9d700 1 -- 172.21.15.66:0/983863668 learned_addr learned my addr 172.21.15.66:0/983863668 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24cf59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 0x7f24d01a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60498/0 (socket says 172.21.15.66:60498) 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24cf59e700 1 -- 172.21.15.66:0/983863668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24d0102790 msgr2=0x7f24d0199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.382+0000 7f24cf59e700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24d0102790 0x7f24d0199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24cf59e700 1 -- 172.21.15.66:0/983863668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 msgr2=0x7f24d01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24cf59e700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 0x7f24d01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24cf59e700 1 -- 172.21.15.66:0/983863668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80095f0 con 0x7f24d0103130 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24ced9d700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 0x7f24d01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24cf59e700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 0x7f24d01a0bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f24c800efc0 tx=0x7f24c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24ccd99700 1 -- 172.21.15.66:0/983863668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8018070 con 0x7f24d0103130 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24ccd99700 1 -- 172.21.15.66:0/983863668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24c800f040 con 0x7f24d0103130 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01a16f0 con 0x7f24d0103130 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24ccd99700 1 -- 172.21.15.66:0/983863668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800cba0 con 0x7f24d0103130 2023-12-11T19:36:00.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.383+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01a1bb0 con 0x7f24d0103130 2023-12-11T19:36:00.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.388+0000 7f24ccd99700 1 -- 172.21.15.66:0/983863668 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f24c8003a00 con 0x7f24d0103130 2023-12-11T19:36:00.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 Processor -- start 2023-12-11T19:36:00.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 -- start start 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc91943d0 0x557bc91947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc9195640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc91960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc90efb50 con 0x557bc9195c20 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc90ed470 con 0x557bc9194d30 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.389+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc90e8d60 con 0x557bc91943d0 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ccd99700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24bc0a1480 0x7f24bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc9195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc9195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41438/0 (socket says 172.21.15.66:41438) 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 -- 172.21.15.66:0/3534179041 learned_addr learned my addr 172.21.15.66:0/3534179041 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24c7fff700 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc91943d0 0x557bc91947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ccd99700 1 -- 172.21.15.66:0/983863668 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24c80cc840 con 0x7f24d0103130 2023-12-11T19:36:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24cf59e700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24bc0a1480 0x7f24bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24c7fff700 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc91960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 -- 172.21.15.66:0/3534179041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc91943d0 msgr2=0x557bc91947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc91943d0 0x557bc91947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 -- 172.21.15.66:0/3534179041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 msgr2=0x557bc91960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc91960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 -- 172.21.15.66:0/3534179041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bc8f65630 con 0x557bc9194d30 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.390+0000 7f24ced9d700 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc9195640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24c0011650 tx=0x7f24c0011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9d7cd44667435e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.391+0000 7f24cf59e700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24bc0a1480 0x7f24bc0a3940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f24c8096620 tx=0x7f24c80cca80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.391+0000 7f24c57fa700 1 -- 172.21.15.66:0/3534179041 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004a30 con 0x557bc9194d30 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.391+0000 7f24c57fa700 1 -- 172.21.15.66:0/3534179041 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24c00054b0 con 0x557bc9194d30 2023-12-11T19:36:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.391+0000 7f24c57fa700 1 -- 172.21.15.66:0/3534179041 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000f7e0 con 0x557bc9194d30 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.391+0000 7f24e8adcb80 1 -- 172.21.15.66:0/3534179041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 msgr2=0x557bc9195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.391+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc9195640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24c0011650 tx=0x7f24c0011960 comp rx=0 tx=0).stop 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 -- 172.21.15.66:0/3534179041 shutdown_connections 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc91960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc91943d0 0x557bc91947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/3534179041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc9195640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 -- 172.21.15.66:0/3534179041 >> 172.21.15.66:0/3534179041 conn(0x557bc90c37e0 msgr2=0x557bc9190d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 -- 172.21.15.66:0/3534179041 shutdown_connections 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 -- 172.21.15.66:0/3534179041 wait complete. 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.392+0000 7f24e8adcb80 1 -- start start 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 Processor -- start 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc919e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc919e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9195c20 0x557bc91a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc90a2560 con 0x557bc91943d0 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc90a2200 con 0x557bc9195c20 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc8f67920 con 0x557bc9194d30 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24e8adcb80 10 client.0 _ll_register_callbacks cb 0x557bc8fcda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc919e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc919e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60522/0 (socket says 172.21.15.66:60522) 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.393+0000 7f24c7fff700 1 -- 172.21.15.66:0/1461365407 learned_addr learned my addr 172.21.15.66:0/1461365407 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24ced9d700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc919e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9195c20 0x557bc91a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 -- 172.21.15.66:0/1461365407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 msgr2=0x557bc919e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc919e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 -- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9195c20 msgr2=0x557bc91a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9195c20 0x557bc91a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 -- 172.21.15.66:0/1461365407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c0009e10 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc919e1b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f24b800b580 tx=0x7f24b8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9074ddf18f85a08 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24b67fc700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24b67fc700 1 -- 172.21.15.66:0/1461365407 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b801d040 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bc9187b40 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.394+0000 7f24b67fc700 1 -- 172.21.15.66:0/1461365407 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24b80056c0 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24b67fc700 1 -- 172.21.15.66:0/1461365407 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b8003780 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 10 client.16206 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557bc8f11de0 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24b67fc700 1 -- 172.21.15.66:0/1461365407 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557bc9187b40 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24b67fc700 1 -- 172.21.15.66:0/1461365407 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f24b800f8e0 con 0x557bc91943d0 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 20 client.16206 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc91a9f30 0x557bc91ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 4 client.16206 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:00.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.395+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bc9163f40 con 0x557bc91a9f30 2023-12-11T19:36:00.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.396+0000 7f24ced9d700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc91a9f30 0x557bc91ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.396+0000 7f24ced9d700 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc91a9f30 0x557bc91ac350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4251cd0d87947907 server_cookie=b03b8e61eb16104a in_seq=0 out_seq=0 2023-12-11T19:36:00.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.396+0000 7f24b67fc700 10 client.16206 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:36:00.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.397+0000 7f24b67fc700 1 -- 172.21.15.66:0/1461365407 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bc9163f40 con 0x557bc91a9f30 2023-12-11T19:36:00.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.397+0000 7f24b67fc700 10 client.16206 handle_command_reply: tid=0 2023-12-11T19:36:00.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.397+0000 7f24e8adcb80 1 client.16206 shutdown 2023-12-11T19:36:00.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 msgr2=0x557bc919e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc919e1b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f24b800b580 tx=0x7f24b8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 shutdown_connections 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc91a9f30 0x557bc91ac350 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9195c20 0x557bc91a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24b67fc700 0 client.16206 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc919e1b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1461365407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc919e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 >> 172.21.15.66:0/1461365407 conn(0x557bc90c37e0 msgr2=0x557bc918fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 shutdown_connections 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1461365407 wait complete. 2023-12-11T19:36:00.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.398+0000 7f24e8adcb80 20 client.16206 trim_cache size 0 max 16384 2023-12-11T19:36:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 Processor -- start 2023-12-11T19:36:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 -- start start 2023-12-11T19:36:00.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 0x557bc91a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc91a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc9198b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc919f440 con 0x557bc9195c20 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc91b1590 con 0x557bc91943d0 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.412+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc9199700 con 0x557bc9194d30 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 0x557bc91a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 0x557bc91a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:41468/0 (socket says 172.21.15.66:41468) 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 -- 172.21.15.66:0/2745254998 learned_addr learned my addr 172.21.15.66:0/2745254998 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc9198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24ced9d700 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc91a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 -- 172.21.15.66:0/2745254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 msgr2=0x557bc91a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc91a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 -- 172.21.15.66:0/2745254998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 msgr2=0x557bc9198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc9198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 -- 172.21.15.66:0/2745254998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bc8efac60 con 0x557bc91943d0 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.413+0000 7f24c7fff700 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 0x557bc91a5de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000c00 tx=0x7f24b8009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77811a021ba24d67 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24c5ffb700 1 -- 172.21.15.66:0/2745254998 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80056c0 con 0x557bc91943d0 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24c5ffb700 1 -- 172.21.15.66:0/2745254998 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24b8028400 con 0x557bc91943d0 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24c5ffb700 1 -- 172.21.15.66:0/2745254998 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8003ae0 con 0x557bc91943d0 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 -- 172.21.15.66:0/2745254998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 msgr2=0x557bc91a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 0x557bc91a5de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000c00 tx=0x7f24b8009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 -- 172.21.15.66:0/2745254998 shutdown_connections 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc91943d0 0x557bc91a5de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc9195c20 0x557bc9198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/2745254998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9194d30 0x557bc91a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 -- 172.21.15.66:0/2745254998 >> 172.21.15.66:0/2745254998 conn(0x557bc90c37e0 msgr2=0x557bc919d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 -- 172.21.15.66:0/2745254998 shutdown_connections 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.414+0000 7f24e8adcb80 1 -- 172.21.15.66:0/2745254998 wait complete. 2023-12-11T19:36:00.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 -- start start 2023-12-11T19:36:00.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 Processor -- start 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc91a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc91a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9195c20 0x557bc91a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc91a4940 con 0x557bc91943d0 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc91a4ab0 con 0x557bc9194d30 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc91a4c70 con 0x557bc9195c20 2023-12-11T19:36:00.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.415+0000 7f24e8adcb80 10 client.0 _ll_register_callbacks cb 0x557bc8fcda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc91a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc91a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60544/0 (socket says 172.21.15.66:60544) 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 -- 172.21.15.66:0/1632312315 learned_addr learned my addr 172.21.15.66:0/1632312315 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24ced9d700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc91a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9195c20 0x557bc91a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 -- 172.21.15.66:0/1632312315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9195c20 msgr2=0x557bc91a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9195c20 0x557bc91a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 -- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 msgr2=0x557bc91a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc91a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 -- 172.21.15.66:0/1632312315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bc8f11de0 con 0x557bc91943d0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24c7fff700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc91a2eb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c010 tx=0x7f24b800bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8149cc80a2fd2ece server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.416+0000 7f24b67fc700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24b67fc700 1 -- 172.21.15.66:0/1632312315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b801d070 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24c00054b0 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24b67fc700 1 -- 172.21.15.66:0/1632312315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24b8000e40 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24b67fc700 1 -- 172.21.15.66:0/1632312315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b800f980 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24e8adcb80 10 client.16212 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557bc8f38250 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24b67fc700 1 -- 172.21.15.66:0/1632312315 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24b8050020 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.417+0000 7f24b67fc700 1 -- 172.21.15.66:0/1632312315 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f24b8028b70 con 0x557bc91943d0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24e8adcb80 10 client.16212 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24e8adcb80 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24e8adcb80 20 client.16212 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc92009c0 0x557bc9202db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24e8adcb80 4 client.16212 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 --> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557bc90aefd0 con 0x557bc92009c0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.418+0000 7f24ced9d700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc92009c0 0x557bc9202db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.419+0000 7f24ced9d700 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc92009c0 0x557bc9202db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2296fad241fe9d7b server_cookie=f63bca5c45d9a2f3 in_seq=0 out_seq=0 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.419+0000 7f24b67fc700 10 client.16212 ms_handle_connect on v2:172.21.15.70:6834/3670150002 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.419+0000 7f24b67fc700 1 -- 172.21.15.66:0/1632312315 <== mds.3 v2:172.21.15.70:6834/3670150002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557bc90aefd0 con 0x557bc92009c0 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.419+0000 7f24b67fc700 10 client.16212 handle_command_reply: tid=0 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.419+0000 7f24e8adcb80 1 client.16212 shutdown 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 msgr2=0x557bc91a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc91a2eb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c010 tx=0x7f24b800bbc0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 shutdown_connections 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557bc9195c20 0x557bc91a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557bc91943d0 0x557bc91a2eb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:6834/3670150002,v1:172.21.15.70:6835/3670150002] conn(0x557bc92009c0 0x557bc9202db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 --2- 172.21.15.66:0/1632312315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x557bc9194d30 0x557bc91a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24b67fc700 0 client.16212 ms_handle_reset on v2:172.21.15.70:6834/3670150002 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 >> 172.21.15.66:0/1632312315 conn(0x557bc90c37e0 msgr2=0x557bc919d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 shutdown_connections 2023-12-11T19:36:00.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 1 -- 172.21.15.66:0/1632312315 wait complete. 2023-12-11T19:36:00.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24e8adcb80 20 client.16212 trim_cache size 0 max 16384 2023-12-11T19:36:00.425 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24bc0a1480 msgr2=0x7f24bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24d51ff700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24bc0a1480 0x7f24bc0a3940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f24c8096620 tx=0x7f24c80cca80 comp rx=0 tx=0).stop 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 msgr2=0x7f24d01a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.420+0000 7f24d51ff700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 0x7f24d01a0bd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f24c800efc0 tx=0x7f24c800c3a0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 shutdown_connections 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f24bc0a1480 0x7f24bc0a3940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24d01015a0 0x7f24d01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f24d0103130 0x7f24d01a0bd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 --2- 172.21.15.66:0/983863668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24d0102790 0x7f24d0199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 >> 172.21.15.66:0/983863668 conn(0x7f24d00fcb10 msgr2=0x7f24d0108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 shutdown_connections 2023-12-11T19:36:00.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.421+0000 7f24d51ff700 1 -- 172.21.15.66:0/983863668 wait complete. 2023-12-11T19:36:00.439 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-11T19:36:00.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 -- 172.21.15.66:0/3622592016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 msgr2=0x7f01780744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 --2- 172.21.15.66:0/3622592016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f01780744b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f016c009990 tx=0x7f016c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 -- 172.21.15.66:0/3622592016 shutdown_connections 2023-12-11T19:36:00.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 --2- 172.21.15.66:0/3622592016 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f017810db70 0x7f017810ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 --2- 172.21.15.66:0/3622592016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0178075610 0x7f0178073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 --2- 172.21.15.66:0/3622592016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f01780744b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 -- 172.21.15.66:0/3622592016 >> 172.21.15.66:0/3622592016 conn(0x7f01780fcb10 msgr2=0x7f01780fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.649+0000 7f017cbff700 1 -- 172.21.15.66:0/3622592016 shutdown_connections 2023-12-11T19:36:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.650+0000 7f017cbff700 1 -- 172.21.15.66:0/3622592016 wait complete. 2023-12-11T19:36:00.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.650+0000 7f017cbff700 1 Processor -- start 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 -- start start 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f017819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 0x7f017819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017810db70 0x7f01781a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01781a55e0 con 0x7f0178075610 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01781a5720 con 0x7f0178074030 2023-12-11T19:36:00.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017cbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01781a58b0 con 0x7f017810db70 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f0175d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 0x7f017819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f0175d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 0x7f017819dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60556/0 (socket says 172.21.15.66:60556) 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f0175d9b700 1 -- 172.21.15.66:0/3386347219 learned_addr learned my addr 172.21.15.66:0/3386347219 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017810db70 0x7f01781a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.651+0000 7f017659c700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f017819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f0175d9b700 1 -- 172.21.15.66:0/3386347219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017810db70 msgr2=0x7f01781a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017810db70 0x7f01781a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f0175d9b700 1 -- 172.21.15.66:0/3386347219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 msgr2=0x7f017819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f017819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f0175d9b700 1 -- 172.21.15.66:0/3386347219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0095f0 con 0x7f0178075610 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f017659c700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f017819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 0x7f017819dff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f016c000c00 tx=0x7f016c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f01677fe700 1 -- 172.21.15.66:0/3386347219 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c01d070 con 0x7f0178075610 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f01677fe700 1 -- 172.21.15.66:0/3386347219 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f016c00bdb0 con 0x7f0178075610 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01781a5b30 con 0x7f0178075610 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.652+0000 7f01677fe700 1 -- 172.21.15.66:0/3386347219 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c022db0 con 0x7f0178075610 2023-12-11T19:36:00.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.653+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01781a5f50 con 0x7f0178075610 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.657+0000 7f01677fe700 1 -- 172.21.15.66:0/3386347219 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f016c022470 con 0x7f0178075610 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.658+0000 7f0190458b80 1 Processor -- start 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.658+0000 7f0190458b80 1 -- start start 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.658+0000 7f0190458b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.658+0000 7f0190458b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f57f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.658+0000 7f0190458b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.658+0000 7f01677fe700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01600a1480 0x7f01600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0190458b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f574eb50 con 0x5612f57f4c20 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0190458b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f574c470 con 0x5612f57f33d0 2023-12-11T19:36:00.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0190458b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f5747d60 con 0x5612f57f3d30 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f01677fe700 1 -- 172.21.15.66:0/3386347219 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f016c0ce350 con 0x7f0178075610 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60560/0 (socket says 172.21.15.66:60560) 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 -- 172.21.15.66:0/3686609475 learned_addr learned my addr 172.21.15.66:0/3686609475 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f017659c700 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f57f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01600a1480 0x7f01600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 -- 172.21.15.66:0/3686609475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 msgr2=0x5612f57f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f57f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 -- 172.21.15.66:0/3686609475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 msgr2=0x5612f57f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0175d9b700 1 -- 172.21.15.66:0/3686609475 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612f55c4630 con 0x5612f57f4c20 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.659+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:00.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.660+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57f50a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f016c097bc0 tx=0x7f016c09b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea6d4debb28068e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.660+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01600a1480 0x7f01600a3940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f01700138b0 tx=0x7f0170009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.660+0000 7f0164ff9700 1 -- 172.21.15.66:0/3686609475 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c00bbf0 con 0x5612f57f4c20 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.660+0000 7f0164ff9700 1 -- 172.21.15.66:0/3686609475 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f016c09bdc0 con 0x5612f57f4c20 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.660+0000 7f0164ff9700 1 -- 172.21.15.66:0/3686609475 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c0cbf20 con 0x5612f57f4c20 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 -- 172.21.15.66:0/3686609475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 msgr2=0x5612f57f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57f50a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f016c097bc0 tx=0x7f016c09b020 comp rx=0 tx=0).stop 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 -- 172.21.15.66:0/3686609475 shutdown_connections 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57f50a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 --2- 172.21.15.66:0/3686609475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f57f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 -- 172.21.15.66:0/3686609475 >> 172.21.15.66:0/3686609475 conn(0x5612f57227e0 msgr2=0x5612f57efd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 -- 172.21.15.66:0/3686609475 shutdown_connections 2023-12-11T19:36:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 -- 172.21.15.66:0/3686609475 wait complete. 2023-12-11T19:36:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 -- start start 2023-12-11T19:36:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.661+0000 7f0190458b80 1 Processor -- start 2023-12-11T19:36:00.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 0x5612f57fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f57fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f4c20 0x5612f58043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f5701560 con 0x5612f57f33d0 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f5701200 con 0x5612f57f3d30 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f55c6920 con 0x5612f57f4c20 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0190458b80 10 client.0 _ll_register_callbacks cb 0x5612f562ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f017659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 0x5612f57fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f017659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 0x5612f57fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60572/0 (socket says 172.21.15.66:60572) 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f017659c700 1 -- 172.21.15.66:0/2472053541 learned_addr learned my addr 172.21.15.66:0/2472053541 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.662+0000 7f0175d9b700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f57fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f0176d9d700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f4c20 0x5612f58043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f017659c700 1 -- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f4c20 msgr2=0x5612f58043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f017659c700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f4c20 0x5612f58043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f017659c700 1 -- 172.21.15.66:0/2472053541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 msgr2=0x5612f57fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f017659c700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f57fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f017659c700 1 -- 172.21.15.66:0/2472053541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01781a5f50 con 0x5612f57f33d0 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f0175d9b700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f57fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f0176d9d700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f4c20 0x5612f58043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f017659c700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 0x5612f57fce60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0168009d10 tx=0x7f0168003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7bb66ae01e3905 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f015dffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f015dffb700 1 -- 172.21.15.66:0/2472053541 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680041d0 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612f57e6b40 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f015dffb700 1 -- 172.21.15.66:0/2472053541 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0168012860 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.663+0000 7f015dffb700 1 -- 172.21.15.66:0/2472053541 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0168012a20 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 10 client.16224 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5612f5570de0 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f015dffb700 1 -- 172.21.15.66:0/2472053541 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5612f57e6b40 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f015dffb700 1 -- 172.21.15.66:0/2472053541 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f016801cc30 con 0x5612f57f33d0 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 10 client.16224 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 10 client.16224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 20 client.16224 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f5808c80 0x5612f580b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 4 client.16224 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:00.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.664+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612f57c2f40 con 0x5612f5808c80 2023-12-11T19:36:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.665+0000 7f0175d9b700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f5808c80 0x5612f580b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.665+0000 7f0175d9b700 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f5808c80 0x5612f580b0a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=655e37008362d7de server_cookie=5cdd5abd9d6cdac7 in_seq=0 out_seq=0 2023-12-11T19:36:00.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.665+0000 7f015dffb700 10 client.16224 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:36:00.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f015dffb700 1 -- 172.21.15.66:0/2472053541 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612f57c2f40 con 0x5612f5808c80 2023-12-11T19:36:00.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f015dffb700 10 client.16224 handle_command_reply: tid=0 2023-12-11T19:36:00.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 client.16224 shutdown 2023-12-11T19:36:00.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 msgr2=0x5612f57fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 0x5612f57fce60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0168009d10 tx=0x7f0168003960 comp rx=0 tx=0).stop 2023-12-11T19:36:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 shutdown_connections 2023-12-11T19:36:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f5808c80 0x5612f580b0a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f4c20 0x5612f58043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f33d0 0x5612f57fce60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f015dffb700 0 client.16224 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:36:00.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 --2- 172.21.15.66:0/2472053541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f57fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 >> 172.21.15.66:0/2472053541 conn(0x5612f57227e0 msgr2=0x5612f57eea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 shutdown_connections 2023-12-11T19:36:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.666+0000 7f0190458b80 1 -- 172.21.15.66:0/2472053541 wait complete. 2023-12-11T19:36:00.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.667+0000 7f0190458b80 20 client.16224 trim_cache size 0 max 16384 2023-12-11T19:36:00.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.680+0000 7f0190458b80 1 Processor -- start 2023-12-11T19:36:00.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.680+0000 7f0190458b80 1 -- start start 2023-12-11T19:36:00.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0190458b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0190458b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f58043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0190458b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57ef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0190458b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f58102f0 con 0x5612f57f4c20 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0190458b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f58060e0 con 0x5612f57f33d0 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0190458b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f57eab50 con 0x5612f57f3d30 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57ef970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60588/0 (socket says 172.21.15.66:60588) 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 -- 172.21.15.66:0/2188327155 learned_addr learned my addr 172.21.15.66:0/2188327155 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f017659c700 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0175d9b700 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f58043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 -- 172.21.15.66:0/2188327155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 msgr2=0x5612f58043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f58043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 -- 172.21.15.66:0/2188327155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 msgr2=0x5612f57f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 -- 172.21.15.66:0/2188327155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612f5559c60 con 0x5612f57f4c20 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.681+0000 7f0176d9d700 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57ef970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f017001a8d0 tx=0x7f0170012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa86d97b6c4861 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f01657fa700 1 -- 172.21.15.66:0/2188327155 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017001f050 con 0x5612f57f4c20 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f01657fa700 1 -- 172.21.15.66:0/2188327155 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0170020040 con 0x5612f57f4c20 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f01657fa700 1 -- 172.21.15.66:0/2188327155 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000e040 con 0x5612f57f4c20 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 -- 172.21.15.66:0/2188327155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 msgr2=0x5612f57ef970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57ef970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f017001a8d0 tx=0x7f0170012000 comp rx=0 tx=0).stop 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 -- 172.21.15.66:0/2188327155 shutdown_connections 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f33d0 0x5612f57f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f57ef970 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 --2- 172.21.15.66:0/2188327155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f3d30 0x5612f58043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 -- 172.21.15.66:0/2188327155 >> 172.21.15.66:0/2188327155 conn(0x5612f57227e0 msgr2=0x5612f57fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.682+0000 7f0190458b80 1 -- 172.21.15.66:0/2188327155 shutdown_connections 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.683+0000 7f0190458b80 1 -- 172.21.15.66:0/2188327155 wait complete. 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.683+0000 7f0190458b80 1 -- start start 2023-12-11T19:36:00.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.683+0000 7f0190458b80 1 Processor -- start 2023-12-11T19:36:00.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.683+0000 7f0190458b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f33d0 0x5612f57fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0190458b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f5800510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0190458b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f58034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0190458b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f5803a20 con 0x5612f57f4c20 2023-12-11T19:36:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0190458b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f5803b90 con 0x5612f57f3d30 2023-12-11T19:36:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0190458b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612f5803d50 con 0x5612f57f33d0 2023-12-11T19:36:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0190458b80 10 client.0 _ll_register_callbacks cb 0x5612f562ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f58034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f58034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60602/0 (socket says 172.21.15.66:60602) 2023-12-11T19:36:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 -- 172.21.15.66:0/3196064456 learned_addr learned my addr 172.21.15.66:0/3196064456 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f017659c700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f33d0 0x5612f57fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f5800510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 -- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f33d0 msgr2=0x5612f57fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f33d0 0x5612f57fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 -- 172.21.15.66:0/3196064456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 msgr2=0x5612f5800510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f5800510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0176d9d700 1 -- 172.21.15.66:0/3196064456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017001a550 con 0x5612f57f4c20 2023-12-11T19:36:00.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.684+0000 7f0175d9b700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f5800510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:00.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f0176d9d700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f58034e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f01700258d0 tx=0x7f0170026e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdde6bcf5379bc24 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f015dffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f015dffb700 1 -- 172.21.15.66:0/3196064456 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000e070 con 0x5612f57f4c20 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f016c09bdc0 con 0x5612f57f4c20 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f015dffb700 1 -- 172.21.15.66:0/3196064456 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0170004e20 con 0x5612f57f4c20 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f015dffb700 1 -- 172.21.15.66:0/3196064456 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017001f030 con 0x5612f57f4c20 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f0190458b80 10 client.16230 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5612f5597250 con 0x5612f57f4c20 2023-12-11T19:36:00.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.685+0000 7f015dffb700 1 -- 172.21.15.66:0/3196064456 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f016c09bdc0 con 0x5612f57f4c20 2023-12-11T19:36:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f015dffb700 1 -- 172.21.15.66:0/3196064456 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0170008680 con 0x5612f57f4c20 2023-12-11T19:36:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f0190458b80 10 client.16230 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f0190458b80 10 client.16230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-12-11T19:36:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f0190458b80 20 client.16230 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f0190458b80 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f585ffa0 0x5612f58623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f0190458b80 4 client.16230 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 --> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612f570dfd0 con 0x5612f585ffa0 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.686+0000 7f017659c700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f585ffa0 0x5612f58623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.687+0000 7f017659c700 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f585ffa0 0x5612f58623c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ddde06b4bfa93ec server_cookie=f480ff8e301397a1 in_seq=0 out_seq=0 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.687+0000 7f015dffb700 10 client.16230 ms_handle_connect on v2:172.21.15.167:6834/1867143564 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.687+0000 7f015dffb700 1 -- 172.21.15.66:0/3196064456 <== mds.2 v2:172.21.15.167:6834/1867143564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612f570dfd0 con 0x5612f585ffa0 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.687+0000 7f015dffb700 10 client.16230 handle_command_reply: tid=0 2023-12-11T19:36:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 client.16230 shutdown 2023-12-11T19:36:00.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 msgr2=0x5612f58034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f58034e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f01700258d0 tx=0x7f0170026e00 comp rx=0 tx=0).stop 2023-12-11T19:36:00.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 shutdown_connections 2023-12-11T19:36:00.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:6834/1867143564,v1:172.21.15.167:6835/1867143564] conn(0x5612f585ffa0 0x5612f58623c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5612f57f4c20 0x5612f58034e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f015dffb700 0 client.16230 ms_handle_reset on v2:172.21.15.167:6834/1867143564 2023-12-11T19:36:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612f57f33d0 0x5612f57fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 --2- 172.21.15.66:0/3196064456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612f57f3d30 0x5612f5800510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 >> 172.21.15.66:0/3196064456 conn(0x5612f57227e0 msgr2=0x5612f57ef970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 shutdown_connections 2023-12-11T19:36:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 1 -- 172.21.15.66:0/3196064456 wait complete. 2023-12-11T19:36:00.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.688+0000 7f0190458b80 20 client.16230 trim_cache size 0 max 16384 2023-12-11T19:36:00.700 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:36:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01600a1480 msgr2=0x7f01600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01600a1480 0x7f01600a3940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f01700138b0 tx=0x7f0170009920 comp rx=0 tx=0).stop 2023-12-11T19:36:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 msgr2=0x7f017819dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 0x7f017819dff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f016c000c00 tx=0x7f016c005740 comp rx=0 tx=0).stop 2023-12-11T19:36:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 shutdown_connections 2023-12-11T19:36:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01600a1480 0x7f01600a3940 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0178074030 0x7f017819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f017810db70 0x7f01781a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 --2- 172.21.15.66:0/3386347219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0178075610 0x7f017819dff0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 >> 172.21.15.66:0/3386347219 conn(0x7f01780fcb10 msgr2=0x7f0178100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 shutdown_connections 2023-12-11T19:36:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.689+0000 7f017cbff700 1 -- 172.21.15.66:0/3386347219 wait complete. 2023-12-11T19:36:00.707 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 rewriters = 210313.55 kB/sec 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 rewriters = 210260.78 kB/sec 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 210313.55 kB/sec 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 210313.55 kB/sec 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 210313.55 kB/sec 2023-12-11T19:36:00.884 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:36:00.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 -- 172.21.15.66:0/1341569690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad8103130 msgr2=0x7faad81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 --2- 172.21.15.66:0/1341569690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad8103130 0x7faad81099e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faad000b0d0 tx=0x7faad000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 -- 172.21.15.66:0/1341569690 shutdown_connections 2023-12-11T19:36:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 --2- 172.21.15.66:0/1341569690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad8103130 0x7faad81099e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 --2- 172.21.15.66:0/1341569690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad81015a0 0x7faad81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 --2- 172.21.15.66:0/1341569690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad8102790 0x7faad8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 -- 172.21.15.66:0/1341569690 >> 172.21.15.66:0/1341569690 conn(0x7faad80fcb10 msgr2=0x7faad80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 -- 172.21.15.66:0/1341569690 shutdown_connections 2023-12-11T19:36:00.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.920+0000 7faadcbff700 1 -- 172.21.15.66:0/1341569690 wait complete. 2023-12-11T19:36:00.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.921+0000 7faadcbff700 1 Processor -- start 2023-12-11T19:36:00.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.921+0000 7faadcbff700 1 -- start start 2023-12-11T19:36:00.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faadcbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 0x7faad8075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faadcbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad8102790 0x7faad80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faadcbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad8103130 0x7faad8073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faadcbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad80746a0 con 0x7faad81015a0 2023-12-11T19:36:00.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faadcbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8074810 con 0x7faad8102790 2023-12-11T19:36:00.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faadcbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a1580 con 0x7faad8103130 2023-12-11T19:36:00.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 0x7faad8075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 0x7faad8075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60612/0 (socket says 172.21.15.66:60612) 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 -- 172.21.15.66:0/175050496 learned_addr learned my addr 172.21.15.66:0/175050496 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad6d9d700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad8103130 0x7faad8073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad5d9b700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad8102790 0x7faad80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 -- 172.21.15.66:0/175050496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad8103130 msgr2=0x7faad8073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad8103130 0x7faad8073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 -- 172.21.15.66:0/175050496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad8102790 msgr2=0x7faad80739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad8102790 0x7faad80739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.922+0000 7faad659c700 1 -- 172.21.15.66:0/175050496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad0009d00 con 0x7faad81015a0 2023-12-11T19:36:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.923+0000 7faad659c700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 0x7faad8075330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faac800d980 tx=0x7faac800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.923+0000 7faac77fe700 1 -- 172.21.15.66:0/175050496 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800bb00 con 0x7faad81015a0 2023-12-11T19:36:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.923+0000 7faac77fe700 1 -- 172.21.15.66:0/175050496 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faac8007620 con 0x7faad81015a0 2023-12-11T19:36:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.923+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad81a1810 con 0x7faad81015a0 2023-12-11T19:36:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.923+0000 7faac77fe700 1 -- 172.21.15.66:0/175050496 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800f3c0 con 0x7faad81015a0 2023-12-11T19:36:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.923+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81a1d20 con 0x7faad81015a0 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.928+0000 7faac77fe700 1 -- 172.21.15.66:0/175050496 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7faac8010460 con 0x7faad81015a0 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 Processor -- start 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 -- start start 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422961d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961dd30 0x56422961e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961ec20 0x56422961f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564229578b50 con 0x56422961d3d0 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564229576470 con 0x56422961dd30 2023-12-11T19:36:00.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.929+0000 7faaf0564b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564229571d60 con 0x56422961ec20 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422961d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422961d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60616/0 (socket says 172.21.15.66:60616) 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad6d9d700 1 -- 172.21.15.66:0/772935584 learned_addr learned my addr 172.21.15.66:0/772935584 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faac77fe700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faac00a1480 0x7faac00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad5d9b700 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961ec20 0x56422961f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad659c700 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961dd30 0x56422961e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faac77fe700 1 -- 172.21.15.66:0/175050496 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faac80cccc0 con 0x7faad81015a0 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad6d9d700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faac00a1480 0x7faac00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad5d9b700 1 -- 172.21.15.66:0/772935584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961dd30 msgr2=0x56422961e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad5d9b700 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961dd30 0x56422961e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad5d9b700 1 -- 172.21.15.66:0/772935584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 msgr2=0x56422961d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad5d9b700 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422961d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.930+0000 7faad5d9b700 1 -- 172.21.15.66:0/772935584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642293ee630 con 0x56422961ec20 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.931+0000 7faad5d9b700 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961ec20 0x56422961f0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faacc009be0 tx=0x7faacc009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0aad04a5a306b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.931+0000 7faad6d9d700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faac00a1480 0x7faac00a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faad0009850 tx=0x7faad000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.931+0000 7faac4ff9700 1 -- 172.21.15.66:0/772935584 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc010b80 con 0x56422961ec20 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.931+0000 7faac4ff9700 1 -- 172.21.15.66:0/772935584 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faacc0044c0 con 0x56422961ec20 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.931+0000 7faac4ff9700 1 -- 172.21.15.66:0/772935584 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc003d60 con 0x56422961ec20 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 -- 172.21.15.66:0/772935584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961ec20 msgr2=0x56422961f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961ec20 0x56422961f0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faacc009be0 tx=0x7faacc009ef0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 -- 172.21.15.66:0/772935584 shutdown_connections 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961ec20 0x56422961f0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422961d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 --2- 172.21.15.66:0/772935584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961dd30 0x56422961e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 -- 172.21.15.66:0/772935584 >> 172.21.15.66:0/772935584 conn(0x56422954c7e0 msgr2=0x564229619d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 -- 172.21.15.66:0/772935584 shutdown_connections 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 -- 172.21.15.66:0/772935584 wait complete. 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.932+0000 7faaf0564b80 1 -- start start 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 Processor -- start 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961d3d0 0x564229627150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x564229627690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 0x56422962e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422952b560 con 0x56422961ec20 2023-12-11T19:36:00.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422952b200 con 0x56422961d3d0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642293f0920 con 0x56422961dd30 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faaf0564b80 10 client.0 _ll_register_callbacks cb 0x564229456a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faad6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 0x56422962e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faad6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 0x56422962e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60630/0 (socket says 172.21.15.66:60630) 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faad6d9d700 1 -- 172.21.15.66:0/3115342164 learned_addr learned my addr 172.21.15.66:0/3115342164 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faad659c700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961d3d0 0x564229627150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.933+0000 7faad5d9b700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x564229627690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad6d9d700 1 -- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 msgr2=0x564229627690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad6d9d700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x564229627690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad6d9d700 1 -- 172.21.15.66:0/3115342164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961d3d0 msgr2=0x564229627150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad6d9d700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961d3d0 0x564229627150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad6d9d700 1 -- 172.21.15.66:0/3115342164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc009810 con 0x56422961ec20 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad659c700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961d3d0 0x564229627150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faad6d9d700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 0x56422962e6d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faad000bcd0 tx=0x7faad00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec2ceb7b44eb85 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faabdffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faabdffb700 1 -- 172.21.15.66:0/3115342164 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0025040 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faabdffb700 1 -- 172.21.15.66:0/3115342164 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faad002b670 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.934+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564229610b40 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faabdffb700 1 -- 172.21.15.66:0/3115342164 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad000e070 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 10 client.16242 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56422939ade0 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faabdffb700 1 -- 172.21.15.66:0/3115342164 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564229610b40 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faabdffb700 1 -- 172.21.15.66:0/3115342164 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faad0005040 con 0x56422961ec20 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 20 client.16242 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229632ec0 0x5642296352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 4 client.16242 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:00.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.935+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642295ecf40 con 0x564229632ec0 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.936+0000 7faad659c700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229632ec0 0x5642296352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.937+0000 7faad659c700 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229632ec0 0x5642296352e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2fb4ee176058a2d server_cookie=f41ca58cb902ce4b in_seq=0 out_seq=0 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.937+0000 7faabdffb700 10 client.16242 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faabdffb700 1 -- 172.21.15.66:0/3115342164 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642295ecf40 con 0x564229632ec0 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faabdffb700 10 client.16242 handle_command_reply: tid=0 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 client.16242 shutdown 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 msgr2=0x56422962e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 0x56422962e6d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faad000bcd0 tx=0x7faad00099e0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 shutdown_connections 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229632ec0 0x5642296352e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961d3d0 0x564229627150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961ec20 0x56422962e6d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 --2- 172.21.15.66:0/3115342164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x564229627690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faabdffb700 0 client.16242 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.938+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 >> 172.21.15.66:0/3115342164 conn(0x56422954c7e0 msgr2=0x564229618a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.939+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 shutdown_connections 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.939+0000 7faaf0564b80 1 -- 172.21.15.66:0/3115342164 wait complete. 2023-12-11T19:36:00.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.939+0000 7faaf0564b80 20 client.16242 trim_cache size 0 max 16384 2023-12-11T19:36:00.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.954+0000 7faaf0564b80 1 Processor -- start 2023-12-11T19:36:00.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.954+0000 7faaf0564b80 1 -- start start 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x564229626660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422961f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faaf0564b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564229626cc0 con 0x56422961d3d0 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faaf0564b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422961e430 con 0x56422961ec20 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faaf0564b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564229624dd0 con 0x56422961dd30 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x564229626660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x564229626660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60638/0 (socket says 172.21.15.66:60638) 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 -- 172.21.15.66:0/2826991413 learned_addr learned my addr 172.21.15.66:0/2826991413 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad6d9d700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422961f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 -- 172.21.15.66:0/2826991413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 msgr2=0x56422962e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad5d9b700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 -- 172.21.15.66:0/2826991413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 msgr2=0x56422961f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422961f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad659c700 1 -- 172.21.15.66:0/2826991413 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564229383c60 con 0x56422961d3d0 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad6d9d700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422961f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.955+0000 7faad5d9b700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.956+0000 7faad659c700 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x564229626660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faac8096a50 tx=0x7faac8015310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aca105bc4effbb server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.956+0000 7faac57fa700 1 -- 172.21.15.66:0/2826991413 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8099fe0 con 0x56422961d3d0 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.956+0000 7faac57fa700 1 -- 172.21.15.66:0/2826991413 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faac809e130 con 0x56422961d3d0 2023-12-11T19:36:00.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.956+0000 7faac57fa700 1 -- 172.21.15.66:0/2826991413 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac80ca8d0 con 0x56422961d3d0 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 -- 172.21.15.66:0/2826991413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 msgr2=0x564229626660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x564229626660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faac8096a50 tx=0x7faac8015310 comp rx=0 tx=0).stop 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 -- 172.21.15.66:0/2826991413 shutdown_connections 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422961f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x564229626660 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 --2- 172.21.15.66:0/2826991413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 -- 172.21.15.66:0/2826991413 >> 172.21.15.66:0/2826991413 conn(0x56422954c7e0 msgr2=0x564229639c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 -- 172.21.15.66:0/2826991413 shutdown_connections 2023-12-11T19:36:00.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 -- 172.21.15.66:0/2826991413 wait complete. 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 -- start start 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.957+0000 7faaf0564b80 1 Processor -- start 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422962a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422962d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422962da20 con 0x56422961d3d0 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422962db90 con 0x56422961ec20 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56422962dd50 con 0x56422961dd30 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faaf0564b80 10 client.0 _ll_register_callbacks cb 0x564229456a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422962a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422962a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60640/0 (socket says 172.21.15.66:60640) 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 -- 172.21.15.66:0/4142120829 learned_addr learned my addr 172.21.15.66:0/4142120829 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 -- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 msgr2=0x56422962a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad6d9d700 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422962d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 -- 172.21.15.66:0/4142120829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 msgr2=0x56422962d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422962d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.958+0000 7faad659c700 1 -- 172.21.15.66:0/4142120829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81a1d20 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faad659c700 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422962a020 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faac8099f70 tx=0x7faac809a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb030d639ebfde85 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faabdffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faabdffb700 1 -- 172.21.15.66:0/4142120829 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800bd60 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faabdffb700 1 -- 172.21.15.66:0/4142120829 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faac8010710 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faacc0044c0 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faabdffb700 1 -- 172.21.15.66:0/4142120829 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac80b8170 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faaf0564b80 10 client.16248 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5642293c1250 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.959+0000 7faabdffb700 1 -- 172.21.15.66:0/4142120829 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faacc0044c0 con 0x56422961d3d0 2023-12-11T19:36:00.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faabdffb700 1 -- 172.21.15.66:0/4142120829 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faac80994d0 con 0x56422961d3d0 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faaf0564b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faaf0564b80 10 client.16248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faaf0564b80 20 client.16248 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faaf0564b80 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229689a60 0x56422968be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faaf0564b80 4 client.16248 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 --> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564229537fd0 con 0x564229689a60 2023-12-11T19:36:00.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.960+0000 7faad5d9b700 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229689a60 0x56422968be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faad5d9b700 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229689a60 0x56422968be80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a68bedaabf13aacf server_cookie=4e0056c9b883895a in_seq=0 out_seq=0 2023-12-11T19:36:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faabdffb700 10 client.16248 ms_handle_connect on v2:172.21.15.167:6838/3906090427 2023-12-11T19:36:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faabdffb700 1 -- 172.21.15.66:0/4142120829 <== mds.1 v2:172.21.15.167:6838/3906090427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564229537fd0 con 0x564229689a60 2023-12-11T19:36:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faabdffb700 10 client.16248 handle_command_reply: tid=0 2023-12-11T19:36:00.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 client.16248 shutdown 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 msgr2=0x56422962a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422962a020 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faac8099f70 tx=0x7faac809a850 comp rx=0 tx=0).stop 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 shutdown_connections 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:6838/3906090427,v1:172.21.15.167:6839/3906090427] conn(0x564229689a60 0x56422968be80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56422961ec20 0x56422962d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56422961d3d0 0x56422962a020 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faabdffb700 0 client.16248 ms_handle_reset on v2:172.21.15.167:6838/3906090427 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 --2- 172.21.15.66:0/4142120829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56422961dd30 0x56422962a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 >> 172.21.15.66:0/4142120829 conn(0x56422954c7e0 msgr2=0x564229621b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 shutdown_connections 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.961+0000 7faaf0564b80 1 -- 172.21.15.66:0/4142120829 wait complete. 2023-12-11T19:36:00.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faaf0564b80 20 client.16248 trim_cache size 0 max 16384 2023-12-11T19:36:00.966 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:36:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faac00a1480 msgr2=0x7faac00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faac00a1480 0x7faac00a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faad0009850 tx=0x7faad000b040 comp rx=0 tx=0).stop 2023-12-11T19:36:00.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 msgr2=0x7faad8075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 0x7faad8075330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faac800d980 tx=0x7faac800dd40 comp rx=0 tx=0).stop 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 shutdown_connections 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faac00a1480 0x7faac00a3940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faad8103130 0x7faad8073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faad81015a0 0x7faad8075330 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 --2- 172.21.15.66:0/175050496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad8102790 0x7faad80739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 >> 172.21.15.66:0/175050496 conn(0x7faad80fcb10 msgr2=0x7faad8105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 shutdown_connections 2023-12-11T19:36:00.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:00.962+0000 7faadcbff700 1 -- 172.21.15.66:0/175050496 wait complete. 2023-12-11T19:36:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:00 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/2009849837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:36:00.981 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-11T19:36:01.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:00 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/2009849837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:36:01.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:00 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/2009849837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T19:36:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 -- 172.21.15.66:0/2532583062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5808102790 msgr2=0x7f5808102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 --2- 172.21.15.66:0/2532583062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5808102790 0x7f5808102bf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f57f00099c0 tx=0x7f57f0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 -- 172.21.15.66:0/2532583062 shutdown_connections 2023-12-11T19:36:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 --2- 172.21.15.66:0/2532583062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58081015a0 0x7f58081019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 --2- 172.21.15.66:0/2532583062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5808103130 0x7f58081099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 --2- 172.21.15.66:0/2532583062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5808102790 0x7f5808102bf0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 -- 172.21.15.66:0/2532583062 >> 172.21.15.66:0/2532583062 conn(0x7f58080fcb10 msgr2=0x7f58080fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:01.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 -- 172.21.15.66:0/2532583062 shutdown_connections 2023-12-11T19:36:01.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.194+0000 7f580cfff700 1 -- 172.21.15.66:0/2532583062 wait complete. 2023-12-11T19:36:01.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 Processor -- start 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 -- start start 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 0x7f5808199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5808102790 0x7f5808199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5808103130 0x7f58081a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081a1220 con 0x7f58081015a0 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081a1360 con 0x7f5808102790 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.195+0000 7f580cfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081a14f0 con 0x7f5808103130 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 0x7f5808199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 0x7f5808199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60666/0 (socket says 172.21.15.66:60666) 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 -- 172.21.15.66:0/2703588632 learned_addr learned my addr 172.21.15.66:0/2703588632 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f57fffff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5808102790 0x7f5808199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 -- 172.21.15.66:0/2703588632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5808103130 msgr2=0x7f58081a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5808103130 0x7f58081a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 -- 172.21.15.66:0/2703588632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5808102790 msgr2=0x7f5808199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5808102790 0x7f5808199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 -- 172.21.15.66:0/2703588632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00095f0 con 0x7f58081015a0 2023-12-11T19:36:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.196+0000 7f580659c700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 0x7f5808199740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f57f800d980 tx=0x7f57f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:01.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.197+0000 7f57ff7fe700 1 -- 172.21.15.66:0/2703588632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8004020 con 0x7f58081015a0 2023-12-11T19:36:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.197+0000 7f57ff7fe700 1 -- 172.21.15.66:0/2703588632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57f8009730 con 0x7f58081015a0 2023-12-11T19:36:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.197+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58081a17d0 con 0x7f58081015a0 2023-12-11T19:36:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.197+0000 7f57ff7fe700 1 -- 172.21.15.66:0/2703588632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003a00 con 0x7f58081015a0 2023-12-11T19:36:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.197+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58081a1ce0 con 0x7f58081015a0 2023-12-11T19:36:01.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.200+0000 7f57ff7fe700 1 -- 172.21.15.66:0/2703588632 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f57f80098a0 con 0x7f58081015a0 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.201+0000 7f57ff7fe700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57f40a1480 0x7f57f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.201+0000 7f58209ebb80 1 Processor -- start 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f57ff7fe700 1 -- 172.21.15.66:0/2703588632 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57f80cca10 con 0x7f58081015a0 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 -- start start 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f57fffff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57f40a1480 0x7f57f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bccd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccdd30 0x558e0bcce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bc28b50 con 0x558e0bccd3d0 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bc26470 con 0x558e0bccdd30 2023-12-11T19:36:01.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f58209ebb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bc21d60 con 0x558e0bccec20 2023-12-11T19:36:01.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f580659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bccd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f57fffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccdd30 0x558e0bcce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f580659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bccd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60668/0 (socket says 172.21.15.66:60668) 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.202+0000 7f580659c700 1 -- 172.21.15.66:0/578708378 learned_addr learned my addr 172.21.15.66:0/578708378 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f57fffff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57f40a1480 0x7f57f40a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b580 tx=0x7f57f0003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 -- 172.21.15.66:0/578708378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccdd30 msgr2=0x558e0bcce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccdd30 0x558e0bcce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 -- 172.21.15.66:0/578708378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 msgr2=0x558e0bccd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bccd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 -- 172.21.15.66:0/578708378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0ba9e630 con 0x558e0bccec20 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f57fffff700 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccdd30 0x558e0bcce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f580659c700 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bccd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.203+0000 7f5806d9d700 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5800009cc0 tx=0x7f580000b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fff7e7f7da0b57 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:01.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f57fcff9700 1 -- 172.21.15.66:0/578708378 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000bcf0 con 0x558e0bccec20 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f57fcff9700 1 -- 172.21.15.66:0/578708378 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580000be50 con 0x558e0bccec20 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f57fcff9700 1 -- 172.21.15.66:0/578708378 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800012710 con 0x558e0bccec20 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 -- 172.21.15.66:0/578708378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 msgr2=0x558e0bccf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5800009cc0 tx=0x7f580000b410 comp rx=0 tx=0).stop 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 -- 172.21.15.66:0/578708378 shutdown_connections 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bccd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 --2- 172.21.15.66:0/578708378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccdd30 0x558e0bcce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.204+0000 7f58209ebb80 1 -- 172.21.15.66:0/578708378 >> 172.21.15.66:0/578708378 conn(0x558e0bbfc7e0 msgr2=0x558e0bcc9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.205+0000 7f58209ebb80 1 -- 172.21.15.66:0/578708378 shutdown_connections 2023-12-11T19:36:01.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.205+0000 7f58209ebb80 1 -- 172.21.15.66:0/578708378 wait complete. 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.205+0000 7f58209ebb80 1 -- start start 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.205+0000 7f58209ebb80 1 Processor -- start 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bcd71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcd7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccec20 0x558e0bcde750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bbdb560 con 0x558e0bccd3d0 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bbdb200 con 0x558e0bccec20 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0baa0920 con 0x558e0bccdd30 2023-12-11T19:36:01.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f58209ebb80 10 client.0 _ll_register_callbacks cb 0x558e0bb06a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bcd71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bcd71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60672/0 (socket says 172.21.15.66:60672) 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 -- 172.21.15.66:0/4041613944 learned_addr learned my addr 172.21.15.66:0/4041613944 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f5806d9d700 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccec20 0x558e0bcde750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 -- 172.21.15.66:0/4041613944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 msgr2=0x558e0bcd7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcd7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 -- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccec20 msgr2=0x558e0bcde750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccec20 0x558e0bcde750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.206+0000 7f580659c700 1 -- 172.21.15.66:0/4041613944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58081a1ce0 con 0x558e0bccd3d0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f580659c700 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bcd71d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f57f8096760 tx=0x7f57f809a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dd306743b559d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f57edffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f57edffb700 1 -- 172.21.15.66:0/4041613944 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f809a150 con 0x558e0bccd3d0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e0bcc0b40 con 0x558e0bccd3d0 2023-12-11T19:36:01.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f57edffb700 1 -- 172.21.15.66:0/4041613944 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57f80a2080 con 0x558e0bccd3d0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f57edffb700 1 -- 172.21.15.66:0/4041613944 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f80991e0 con 0x558e0bccd3d0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f58209ebb80 10 client.16260 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f58000098f0 con 0x558e0bccd3d0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f57edffb700 1 -- 172.21.15.66:0/4041613944 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558e0bcc0b40 con 0x558e0bccd3d0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.207+0000 7f57edffb700 1 -- 172.21.15.66:0/4041613944 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f57f80a1d10 con 0x558e0bccd3d0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f58209ebb80 10 client.16260 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f58209ebb80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f58209ebb80 20 client.16260 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f58209ebb80 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bce3020 0x558e0bce5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f58209ebb80 4 client.16260 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e0bc9cf40 con 0x558e0bce3020 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.208+0000 7f57fffff700 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bce3020 0x558e0bce5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.209+0000 7f57fffff700 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bce3020 0x558e0bce5440 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3fb314f90ee744 server_cookie=edcf26e0bec9e476 in_seq=0 out_seq=0 2023-12-11T19:36:01.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.209+0000 7f57edffb700 10 client.16260 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:01.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f57edffb700 1 -- 172.21.15.66:0/4041613944 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e0bc9cf40 con 0x558e0bce3020 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f57edffb700 10 client.16260 handle_command_reply: tid=0 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 client.16260 shutdown 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 msgr2=0x558e0bcd71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bcd71d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f57f8096760 tx=0x7f57f809a3f0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 shutdown_connections 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bce3020 0x558e0bce5440 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccec20 0x558e0bcde750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccd3d0 0x558e0bcd71d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 --2- 172.21.15.66:0/4041613944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcd7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 >> 172.21.15.66:0/4041613944 conn(0x558e0bbfc7e0 msgr2=0x558e0bcc8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f57edffb700 0 client.16260 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 shutdown_connections 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 1 -- 172.21.15.66:0/4041613944 wait complete. 2023-12-11T19:36:01.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.210+0000 7f58209ebb80 20 client.16260 trim_cache size 0 max 16384 2023-12-11T19:36:01.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.224+0000 7f58209ebb80 1 Processor -- start 2023-12-11T19:36:01.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.224+0000 7f58209ebb80 1 -- start start 2023-12-11T19:36:01.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 0x558e0bcde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f58209ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcc4b80 con 0x558e0bccdd30 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f58209ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcea610 con 0x558e0bccd3d0 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f58209ebb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcd26f0 con 0x558e0bccec20 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f57fffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 0x558e0bcde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f57fffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 0x558e0bcde710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60688/0 (socket says 172.21.15.66:60688) 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f57fffff700 1 -- 172.21.15.66:0/2693747287 learned_addr learned my addr 172.21.15.66:0/2693747287 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f580659c700 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.225+0000 7f5806d9d700 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fffff700 1 -- 172.21.15.66:0/2693747287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 msgr2=0x558e0bccf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fffff700 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fffff700 1 -- 172.21.15.66:0/2693747287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 msgr2=0x558e0bcdee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fffff700 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fffff700 1 -- 172.21.15.66:0/2693747287 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0ba4ade0 con 0x558e0bccdd30 2023-12-11T19:36:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fffff700 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 0x558e0bcde710 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005950 tx=0x7f57f001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20493364caa350c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fd7fa700 1 -- 172.21.15.66:0/2693747287 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0031650 con 0x558e0bccdd30 2023-12-11T19:36:01.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fd7fa700 1 -- 172.21.15.66:0/2693747287 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57f001e040 con 0x558e0bccdd30 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.226+0000 7f57fd7fa700 1 -- 172.21.15.66:0/2693747287 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004d50 con 0x558e0bccdd30 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 -- 172.21.15.66:0/2693747287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 msgr2=0x558e0bcde710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 0x558e0bcde710 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005950 tx=0x7f57f001a050 comp rx=0 tx=0).stop 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 -- 172.21.15.66:0/2693747287 shutdown_connections 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccec20 0x558e0bccf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 --2- 172.21.15.66:0/2693747287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccdd30 0x558e0bcde710 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 -- 172.21.15.66:0/2693747287 >> 172.21.15.66:0/2693747287 conn(0x558e0bbfc7e0 msgr2=0x558e0bcd5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 -- 172.21.15.66:0/2693747287 shutdown_connections 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 -- 172.21.15.66:0/2693747287 wait complete. 2023-12-11T19:36:01.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.227+0000 7f58209ebb80 1 -- start start 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 Processor -- start 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcdcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 0x558e0bcdda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcddfa0 con 0x558e0bccec20 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcde110 con 0x558e0bccd3d0 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0bcde2d0 con 0x558e0bccdd30 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f58209ebb80 10 client.0 _ll_register_callbacks cb 0x558e0bb06a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f5806d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 0x558e0bcdda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f5806d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 0x558e0bcdda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60694/0 (socket says 172.21.15.66:60694) 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.228+0000 7f5806d9d700 1 -- 172.21.15.66:0/1981642968 learned_addr learned my addr 172.21.15.66:0/1981642968 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f57fffff700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcdcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f580659c700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f5806d9d700 1 -- 172.21.15.66:0/1981642968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 msgr2=0x558e0bcdcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f5806d9d700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcdcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f5806d9d700 1 -- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 msgr2=0x558e0bcdc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f5806d9d700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f5806d9d700 1 -- 172.21.15.66:0/1981642968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580810a330 con 0x558e0bccec20 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f57fffff700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcdcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f580659c700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:01.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f5806d9d700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 0x558e0bcdda60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5800009cc0 tx=0x7f5800013b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627cd5815958678e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.229+0000 7f57edffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f57edffb700 1 -- 172.21.15.66:0/1981642968 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000bcf0 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f580000be50 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f57edffb700 1 -- 172.21.15.66:0/1981642968 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f580001d440 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f57edffb700 1 -- 172.21.15.66:0/1981642968 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f580001d600 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f58209ebb80 10 client.16266 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e0ba33c60 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f57edffb700 1 -- 172.21.15.66:0/1981642968 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f580000be50 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f57edffb700 1 -- 172.21.15.66:0/1981642968 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5800025030 con 0x558e0bccec20 2023-12-11T19:36:01.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f58209ebb80 10 client.16266 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:01.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f58209ebb80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:01.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.230+0000 7f58209ebb80 20 client.16266 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:01.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.231+0000 7f58209ebb80 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bd3a560 0x558e0bd3c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:01.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.231+0000 7f58209ebb80 4 client.16266 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-11T19:36:01.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.231+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e0bbe7fd0 con 0x558e0bd3a560 2023-12-11T19:36:01.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.231+0000 7f580659c700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bd3a560 0x558e0bd3c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:01.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.231+0000 7f57edffb700 10 client.16266 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:01.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.231+0000 7f580659c700 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bd3a560 0x558e0bd3c980 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f62d9c8000488ba7 server_cookie=44d51d6c73aa5411 in_seq=0 out_seq=0 2023-12-11T19:36:01.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f57edffb700 1 -- 172.21.15.66:0/1981642968 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e0bbe7fd0 con 0x558e0bd3a560 2023-12-11T19:36:01.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f57edffb700 10 client.16266 handle_command_reply: tid=0 2023-12-11T19:36:01.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 client.16266 shutdown 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 msgr2=0x558e0bcdda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 0x558e0bcdda60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5800009cc0 tx=0x7f5800013b00 comp rx=0 tx=0).stop 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 shutdown_connections 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558e0bd3a560 0x558e0bd3c980 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558e0bccd3d0 0x558e0bcdc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e0bccec20 0x558e0bcdda60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 --2- 172.21.15.66:0/1981642968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e0bccdd30 0x558e0bcdcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f57edffb700 0 client.16266 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 >> 172.21.15.66:0/1981642968 conn(0x558e0bbfc7e0 msgr2=0x558e0bcd65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 shutdown_connections 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 1 -- 172.21.15.66:0/1981642968 wait complete. 2023-12-11T19:36:01.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.232+0000 7f58209ebb80 20 client.16266 trim_cache size 0 max 16384 2023-12-11T19:36:01.238 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-12-11T19:36:01.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57f40a1480 msgr2=0x7f57f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57f40a1480 0x7f57f40a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b580 tx=0x7f57f0003720 comp rx=0 tx=0).stop 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 msgr2=0x7f5808199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 0x7f5808199740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f57f800d980 tx=0x7f57f800dc90 comp rx=0 tx=0).stop 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 shutdown_connections 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f57f40a1480 0x7f57f40a3940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5808103130 0x7f58081a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58081015a0 0x7f5808199740 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 --2- 172.21.15.66:0/2703588632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5808102790 0x7f5808199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 >> 172.21.15.66:0/2703588632 conn(0x7f58080fcb10 msgr2=0x7f5808107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 shutdown_connections 2023-12-11T19:36:01.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:36:01.233+0000 7f580cfff700 1 -- 172.21.15.66:0/2703588632 wait complete. 2023-12-11T19:36:01.251 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 1807753778, 'ino': 1099511627783, 'path': '/volumes/_nogroup/sv_0/d7198378-99a5-4b89-915f-9be6900c3106/client.0/tmp/f1'}] 2023-12-11T19:36:01.251 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-12-11T19:36:02.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:01 smithi167 ceph-mon[102750]: pgmap v433: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 150 MiB/s wr, 76 op/s 2023-12-11T19:36:02.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:01 smithi066 ceph-mon[97920]: pgmap v433: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 150 MiB/s wr, 76 op/s 2023-12-11T19:36:02.110 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:01 smithi070 ceph-mon[104044]: pgmap v433: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 150 MiB/s wr, 76 op/s 2023-12-11T19:36:03.675 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:03.676 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 readers = 5470106.50 kB/sec 2023-12-11T19:36:03.676 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 readers = 5444091.68 kB/sec 2023-12-11T19:36:03.676 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 5470106.50 kB/sec 2023-12-11T19:36:03.676 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 5470106.50 kB/sec 2023-12-11T19:36:03.676 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 5470106.50 kB/sec 2023-12-11T19:36:03.676 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:36:04.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:03 smithi167 ceph-mon[102750]: pgmap v434: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 168 MiB/s wr, 77 op/s 2023-12-11T19:36:04.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:03 smithi066 ceph-mon[97920]: pgmap v434: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 168 MiB/s wr, 77 op/s 2023-12-11T19:36:04.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:03 smithi070 ceph-mon[104044]: pgmap v434: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 168 MiB/s wr, 77 op/s 2023-12-11T19:36:06.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:05 smithi167 ceph-mon[102750]: pgmap v435: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 168 MiB/s wr, 75 op/s 2023-12-11T19:36:06.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:05 smithi066 ceph-mon[97920]: pgmap v435: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 168 MiB/s wr, 75 op/s 2023-12-11T19:36:06.064 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:06.064 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 re-readers = 3703643.75 kB/sec 2023-12-11T19:36:06.065 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 re-readers = 3691650.00 kB/sec 2023-12-11T19:36:06.065 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 3703643.75 kB/sec 2023-12-11T19:36:06.065 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 3703643.75 kB/sec 2023-12-11T19:36:06.065 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 3703643.75 kB/sec 2023-12-11T19:36:06.065 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 1048576.00 kB 2023-12-11T19:36:06.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:05 smithi070 ceph-mon[104044]: pgmap v435: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 168 MiB/s wr, 75 op/s 2023-12-11T19:36:08.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:07 smithi167 ceph-mon[102750]: pgmap v436: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 120 MiB/s wr, 51 op/s 2023-12-11T19:36:08.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:07 smithi066 ceph-mon[97920]: pgmap v436: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 120 MiB/s wr, 51 op/s 2023-12-11T19:36:08.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:07 smithi070 ceph-mon[104044]: pgmap v436: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 120 MiB/s wr, 51 op/s 2023-12-11T19:36:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:08 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:36:09.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:08 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:36:09.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:08 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:36:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:09 smithi167 ceph-mon[102750]: pgmap v437: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 88 MiB/s wr, 33 op/s 2023-12-11T19:36:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:36:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:36:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:09 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:36:10.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:09 smithi066 ceph-mon[97920]: pgmap v437: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 88 MiB/s wr, 33 op/s 2023-12-11T19:36:10.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:36:10.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:36:10.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:09 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:36:10.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:09 smithi070 ceph-mon[104044]: pgmap v437: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 88 MiB/s wr, 33 op/s 2023-12-11T19:36:10.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:36:10.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:36:10.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:09 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 -- 172.21.15.66:0/2945154961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107200 msgr2=0x7f5090109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 --2- 172.21.15.66:0/2945154961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107200 0x7f5090109670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f508800b0a0 tx=0x7f508800b3b0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 -- 172.21.15.66:0/2945154961 shutdown_connections 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 --2- 172.21.15.66:0/2945154961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107200 0x7f5090109670 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 --2- 172.21.15.66:0/2945154961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5090101fa0 0x7f5090104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 --2- 172.21.15.66:0/2945154961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50901048d0 0x7f5090106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.543+0000 7f50957ff700 1 -- 172.21.15.66:0/2945154961 >> 172.21.15.66:0/2945154961 conn(0x7f50900fb990 msgr2=0x7f50900fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.544+0000 7f50957ff700 1 -- 172.21.15.66:0/2945154961 shutdown_connections 2023-12-11T19:36:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.544+0000 7f50957ff700 1 -- 172.21.15.66:0/2945154961 wait complete. 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 Processor -- start 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 -- start start 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5090101fa0 0x7f5090197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 0x7f50901979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5090107200 0x7f509019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509019efa0 con 0x7f50901048d0 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509019f0e0 con 0x7f5090101fa0 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f50957ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509019f270 con 0x7f5090107200 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f5087fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 0x7f50901979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f5087fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 0x7f50901979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40084/0 (socket says 172.21.15.66:40084) 2023-12-11T19:36:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f5087fff700 1 -- 172.21.15.66:0/4051497811 learned_addr learned my addr 172.21.15.66:0/4051497811 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f508effd700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5090101fa0 0x7f5090197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.545+0000 7f508f7fe700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5090107200 0x7f509019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f5087fff700 1 -- 172.21.15.66:0/4051497811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5090107200 msgr2=0x7f509019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f5087fff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5090107200 0x7f509019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f5087fff700 1 -- 172.21.15.66:0/4051497811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5090101fa0 msgr2=0x7f5090197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f5087fff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5090101fa0 0x7f5090197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f5087fff700 1 -- 172.21.15.66:0/4051497811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088009d00 con 0x7f50901048d0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f5087fff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 0x7f50901979b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f507800e9a0 tx=0x7f507800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f508cff9700 1 -- 172.21.15.66:0/4051497811 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078009790 con 0x7f50901048d0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509019f550 con 0x7f50901048d0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f508cff9700 1 -- 172.21.15.66:0/4051497811 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5078010430 con 0x7f50901048d0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.546+0000 7f508cff9700 1 -- 172.21.15.66:0/4051497811 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507800cea0 con 0x7f50901048d0 2023-12-11T19:36:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.547+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509019fab0 con 0x7f50901048d0 2023-12-11T19:36:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.550+0000 7f508cff9700 1 -- 172.21.15.66:0/4051497811 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f50780105a0 con 0x7f50901048d0 2023-12-11T19:36:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 Processor -- start 2023-12-11T19:36:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 -- start start 2023-12-11T19:36:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f508cff9700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f507c0a1500 0x7f507c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e64ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599dbeaf0 con 0x559599e62ed0 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599dbc410 con 0x559599e63830 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.552+0000 7f50a9063b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599db7d00 con 0x559599e64720 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508cff9700 1 -- 172.21.15.66:0/4051497811 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5078014070 con 0x7f50901048d0 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508effd700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f507c0a1500 0x7f507c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40098/0 (socket says 172.21.15.66:40098) 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508f7fe700 1 -- 172.21.15.66:0/809214685 learned_addr learned my addr 172.21.15.66:0/809214685 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f5087fff700 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508f7fe700 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e64ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f5087fff700 1 -- 172.21.15.66:0/809214685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 msgr2=0x559599e64ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f5087fff700 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e64ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f5087fff700 1 -- 172.21.15.66:0/809214685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 msgr2=0x559599e632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f5087fff700 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f5087fff700 1 -- 172.21.15.66:0/809214685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559599c345d0 con 0x559599e63830 2023-12-11T19:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.553+0000 7f508f7fe700 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.554+0000 7f5087fff700 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e64140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f507800e970 tx=0x7f5078099c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362ebd6b94717e73 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.554+0000 7f508effd700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f507c0a1500 0x7f507c0a39c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5080009c60 tx=0x7f5080009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.555+0000 7f50857fa700 1 -- 172.21.15.66:0/809214685 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078077a30 con 0x559599e63830 2023-12-11T19:36:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.555+0000 7f50857fa700 1 -- 172.21.15.66:0/809214685 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5078077850 con 0x559599e63830 2023-12-11T19:36:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.555+0000 7f50857fa700 1 -- 172.21.15.66:0/809214685 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780774d0 con 0x559599e63830 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 -- 172.21.15.66:0/809214685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 msgr2=0x559599e64140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e64140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f507800e970 tx=0x7f5078099c80 comp rx=0 tx=0).stop 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 -- 172.21.15.66:0/809214685 shutdown_connections 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e64ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 --2- 172.21.15.66:0/809214685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e64140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 -- 172.21.15.66:0/809214685 >> 172.21.15.66:0/809214685 conn(0x559599d92780 msgr2=0x559599e5f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 -- 172.21.15.66:0/809214685 shutdown_connections 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.556+0000 7f50a9063b80 1 -- 172.21.15.66:0/809214685 wait complete. 2023-12-11T19:36:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.557+0000 7f50a9063b80 1 -- start start 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.557+0000 7f50a9063b80 1 Processor -- start 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e74260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599d71500 con 0x559599e62ed0 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599d711a0 con 0x559599e63830 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599c368c0 con 0x559599e64720 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f50a9063b80 10 client.0 _ll_register_callbacks cb 0x559599c9ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f508effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f508effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e6cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40112/0 (socket says 172.21.15.66:40112) 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f508effd700 1 -- 172.21.15.66:0/1111454560 learned_addr learned my addr 172.21.15.66:0/1111454560 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.558+0000 7f5087fff700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508f7fe700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e74260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508effd700 1 -- 172.21.15.66:0/1111454560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 msgr2=0x559599e74260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508effd700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e74260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508effd700 1 -- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 msgr2=0x559599e6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508effd700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e6d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508effd700 1 -- 172.21.15.66:0/1111454560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509010a260 con 0x559599e62ed0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f508effd700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e6cef0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5080009b20 tx=0x7f50800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67969003fd81f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f50767fc700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f50767fc700 1 -- 172.21.15.66:0/1111454560 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080004220 con 0x559599e62ed0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559599e56730 con 0x559599e62ed0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.559+0000 7f50767fc700 1 -- 172.21.15.66:0/1111454560 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5080023070 con 0x559599e62ed0 2023-12-11T19:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50767fc700 1 -- 172.21.15.66:0/1111454560 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5080004380 con 0x559599e62ed0 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 10 client.16278 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f509019fab0 con 0x559599e62ed0 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50767fc700 1 -- 172.21.15.66:0/1111454560 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559599e56730 con 0x559599e62ed0 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50767fc700 1 -- 172.21.15.66:0/1111454560 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5080003b90 con 0x559599e62ed0 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 20 client.16278 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599e78d80 0x559599e7b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 4 client.16278 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.560+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559599e32ee0 con 0x559599e78d80 2023-12-11T19:36:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.561+0000 7f5087fff700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599e78d80 0x559599e7b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.561+0000 7f5087fff700 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599e78d80 0x559599e7b1a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3a81fb59701377 server_cookie=b66c4c5b2663ad43 in_seq=0 out_seq=0 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.561+0000 7f50767fc700 10 client.16278 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50767fc700 1 -- 172.21.15.66:0/1111454560 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559599e32ee0 con 0x559599e78d80 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50767fc700 10 client.16278 handle_command_reply: tid=0 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 client.16278 shutdown 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 msgr2=0x559599e6cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e6cef0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5080009b20 tx=0x7f50800047c0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 shutdown_connections 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599e78d80 0x559599e7b1a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e64720 0x559599e74260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50767fc700 0 client.16278 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e6cef0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 --2- 172.21.15.66:0/1111454560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.562+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 >> 172.21.15.66:0/1111454560 conn(0x559599d92780 msgr2=0x559599e5dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.563+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 shutdown_connections 2023-12-11T19:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.563+0000 7f50a9063b80 1 -- 172.21.15.66:0/1111454560 wait complete. 2023-12-11T19:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.563+0000 7f50a9063b80 20 client.16278 trim_cache size 0 max 16384 2023-12-11T19:36:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 Processor -- start 2023-12-11T19:36:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 -- start start 2023-12-11T19:36:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e62ed0 0x559599e64b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 0x559599e6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599e5ee70 con 0x559599e64720 2023-12-11T19:36:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599e74840 con 0x559599e63830 2023-12-11T19:36:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.579+0000 7f50a9063b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599e6a9b0 con 0x559599e62ed0 2023-12-11T19:36:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 0x559599e6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 0x559599e6bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40114/0 (socket says 172.21.15.66:40114) 2023-12-11T19:36:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 -- 172.21.15.66:0/2867887919 learned_addr learned my addr 172.21.15.66:0/2867887919 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f5087fff700 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508effd700 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e62ed0 0x559599e64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 -- 172.21.15.66:0/2867887919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e62ed0 msgr2=0x559599e64b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e62ed0 0x559599e64b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 -- 172.21.15.66:0/2867887919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 msgr2=0x559599e74220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e74220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 -- 172.21.15.66:0/2867887919 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559599be0d80 con 0x559599e64720 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.580+0000 7f508f7fe700 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 0x559599e6bec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f508800b070 tx=0x7f508800b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e542593738f108d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f5085ffb700 1 -- 172.21.15.66:0/2867887919 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800e040 con 0x559599e64720 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f5085ffb700 1 -- 172.21.15.66:0/2867887919 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50880094c0 con 0x559599e64720 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f5085ffb700 1 -- 172.21.15.66:0/2867887919 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508801b450 con 0x559599e64720 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 -- 172.21.15.66:0/2867887919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 msgr2=0x559599e6bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 0x559599e6bec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f508800b070 tx=0x7f508800b720 comp rx=0 tx=0).stop 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 -- 172.21.15.66:0/2867887919 shutdown_connections 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e64720 0x559599e6bec0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e62ed0 0x559599e64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 --2- 172.21.15.66:0/2867887919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e63830 0x559599e74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 -- 172.21.15.66:0/2867887919 >> 172.21.15.66:0/2867887919 conn(0x559599d92780 msgr2=0x559599e6ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:11.589 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 -- 172.21.15.66:0/2867887919 shutdown_connections 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.581+0000 7f50a9063b80 1 -- 172.21.15.66:0/2867887919 wait complete. 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 -- start start 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 Processor -- start 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e72040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e63830 0x559599e72580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e64720 0x559599e73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599ecd260 con 0x559599e62ed0 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599ecd3d0 con 0x559599e64720 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599e73ad0 con 0x559599e63830 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.582+0000 7f50a9063b80 10 client.0 _ll_register_callbacks cb 0x559599c9ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e72040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e72040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:40116/0 (socket says 172.21.15.66:40116) 2023-12-11T19:36:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 -- 172.21.15.66:0/70474120 learned_addr learned my addr 172.21.15.66:0/70474120 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508f7fe700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e64720 0x559599e73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f5087fff700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e63830 0x559599e72580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 -- 172.21.15.66:0/70474120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e63830 msgr2=0x559599e72580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e63830 0x559599e72580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 -- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e64720 msgr2=0x559599e73590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e64720 0x559599e73590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 -- 172.21.15.66:0/70474120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5088007b90 con 0x559599e62ed0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f5087fff700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e63830 0x559599e72580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f508effd700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e72040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5080003ef0 tx=0x7f5080027c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecee573e73a6970 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.583+0000 7f50767fc700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50767fc700 1 -- 172.21.15.66:0/70474120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508001cdd0 con 0x559599e62ed0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559599d7df70 con 0x559599e62ed0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50767fc700 1 -- 172.21.15.66:0/70474120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5080023040 con 0x559599e62ed0 2023-12-11T19:36:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50767fc700 1 -- 172.21.15.66:0/70474120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5080003680 con 0x559599e62ed0 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 10 client.16284 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559599bc9c00 con 0x559599e62ed0 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50767fc700 1 -- 172.21.15.66:0/70474120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559599d7df70 con 0x559599e62ed0 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50767fc700 1 -- 172.21.15.66:0/70474120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5080004460 con 0x559599e62ed0 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 20 client.16284 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599ed8be0 0x559599edb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 4 client.16284 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:36:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.584+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559599d70430 con 0x559599ed8be0 2023-12-11T19:36:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.585+0000 7f5087fff700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599ed8be0 0x559599edb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.585+0000 7f5087fff700 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599ed8be0 0x559599edb000 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479d03818c2f97fd server_cookie=d738656192319fef in_seq=0 out_seq=0 2023-12-11T19:36:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.585+0000 7f50767fc700 10 client.16284 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.598+0000 7f50767fc700 1 -- 172.21.15.66:0/70474120 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559599d70430 con 0x559599ed8be0 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.598+0000 7f50767fc700 10 client.16284 handle_command_reply: tid=0 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.598+0000 7f50a9063b80 1 client.16284 shutdown 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 msgr2=0x559599e72040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e72040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5080003ef0 tx=0x7f5080027c80 comp rx=0 tx=0).stop 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 shutdown_connections 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x559599ed8be0 0x559599edb000 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x559599e64720 0x559599e73590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559599e62ed0 0x559599e72040 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50767fc700 0 client.16284 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 --2- 172.21.15.66:0/70474120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559599e63830 0x559599e72580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 >> 172.21.15.66:0/70474120 conn(0x559599d92780 msgr2=0x559599e6bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 shutdown_connections 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 1 -- 172.21.15.66:0/70474120 wait complete. 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.599+0000 7f50a9063b80 20 client.16284 trim_cache size 0 max 16384 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f507c0a1500 msgr2=0x7f507c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f507c0a1500 0x7f507c0a39c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5080009c60 tx=0x7f5080009260 comp rx=0 tx=0).stop 2023-12-11T19:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 msgr2=0x7f50901979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 0x7f50901979b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f507800e9a0 tx=0x7f507800ecb0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 shutdown_connections 2023-12-11T19:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f507c0a1500 0x7f507c0a39c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5090101fa0 0x7f5090197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5090107200 0x7f509019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 --2- 172.21.15.66:0/4051497811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50901048d0 0x7f50901979b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 >> 172.21.15.66:0/4051497811 conn(0x7f50900fb990 msgr2=0x7f5090102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 shutdown_connections 2023-12-11T19:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:11.600+0000 7f50957ff700 1 -- 172.21.15.66:0/4051497811 wait complete. 2023-12-11T19:36:12.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:11 smithi167 ceph-mon[102750]: pgmap v438: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 62 MiB/s wr, 29 op/s 2023-12-11T19:36:12.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:11 smithi066 ceph-mon[97920]: pgmap v438: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 62 MiB/s wr, 29 op/s 2023-12-11T19:36:12.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:11 smithi070 ceph-mon[104044]: pgmap v438: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 62 MiB/s wr, 29 op/s 2023-12-11T19:36:12.521 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:12.521 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:12.521 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:12.522 INFO:tasks.workunit.client.0.smithi066.stdout:iozone test complete. 2023-12-11T19:36:12.525 INFO:tasks.workunit.client.0.smithi066.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Iozone: Performance Test of File I/O 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Version $Revision: 3.489 $ 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Compiled for 64 bit mode. 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Build: linux-AMD64 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2023-12-11T19:36:12.577 INFO:tasks.workunit.client.0.smithi066.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Alexey Skidanov, Sudhir Kumar. 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Run began: Mon Dec 11 19:36:12 2023 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Include close in write timing 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Include fsync in write timing 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: File size set to 10485760 kB 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Record Size 1024 kB 2023-12-11T19:36:12.578 INFO:tasks.workunit.client.0.smithi066.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: Output is in kBytes/sec 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: Time Resolution = 0.000001 seconds. 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: Processor cache size set to 1024 kBytes. 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: Processor cache line size set to 32 bytes. 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: File stride size set to 17 * record size. 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: Throughput test with 1 process 2023-12-11T19:36:12.579 INFO:tasks.workunit.client.0.smithi066.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2023-12-11T19:36:14.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:13 smithi167 ceph-mon[102750]: pgmap v439: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 15 op/s 2023-12-11T19:36:14.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:13 smithi066 ceph-mon[97920]: pgmap v439: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 15 op/s 2023-12-11T19:36:14.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:13 smithi070 ceph-mon[104044]: pgmap v439: 97 pgs: 97 active+clean; 1.0 GiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 15 op/s 2023-12-11T19:36:16.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:15 smithi167 ceph-mon[102750]: pgmap v440: 97 pgs: 97 active+clean; 673 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 MiB/s wr, 26 op/s 2023-12-11T19:36:16.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:15 smithi066 ceph-mon[97920]: pgmap v440: 97 pgs: 97 active+clean; 673 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 MiB/s wr, 26 op/s 2023-12-11T19:36:16.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:15 smithi070 ceph-mon[104044]: pgmap v440: 97 pgs: 97 active+clean; 673 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 MiB/s wr, 26 op/s 2023-12-11T19:36:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:17 smithi167 ceph-mon[102750]: pgmap v441: 97 pgs: 97 active+clean; 741 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 50 MiB/s wr, 47 op/s 2023-12-11T19:36:18.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:17 smithi066 ceph-mon[97920]: pgmap v441: 97 pgs: 97 active+clean; 741 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 50 MiB/s wr, 47 op/s 2023-12-11T19:36:18.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:17 smithi070 ceph-mon[104044]: pgmap v441: 97 pgs: 97 active+clean; 741 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 50 MiB/s wr, 47 op/s 2023-12-11T19:36:20.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:19 smithi167 ceph-mon[102750]: pgmap v442: 97 pgs: 97 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 88 MiB/s wr, 57 op/s 2023-12-11T19:36:20.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:19 smithi066 ceph-mon[97920]: pgmap v442: 97 pgs: 97 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 88 MiB/s wr, 57 op/s 2023-12-11T19:36:20.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:19 smithi070 ceph-mon[104044]: pgmap v442: 97 pgs: 97 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 88 MiB/s wr, 57 op/s 2023-12-11T19:36:22.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:21 smithi167 ceph-mon[102750]: pgmap v443: 97 pgs: 97 active+clean; 2.0 GiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 167 MiB/s wr, 80 op/s 2023-12-11T19:36:22.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:21 smithi066 ceph-mon[97920]: pgmap v443: 97 pgs: 97 active+clean; 2.0 GiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 167 MiB/s wr, 80 op/s 2023-12-11T19:36:22.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:21 smithi070 ceph-mon[104044]: pgmap v443: 97 pgs: 97 active+clean; 2.0 GiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 167 MiB/s wr, 80 op/s 2023-12-11T19:36:24.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:23 smithi066 ceph-mon[97920]: pgmap v444: 97 pgs: 97 active+clean; 2.4 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 206 MiB/s wr, 88 op/s 2023-12-11T19:36:24.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:23 smithi070 ceph-mon[104044]: pgmap v444: 97 pgs: 97 active+clean; 2.4 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 206 MiB/s wr, 88 op/s 2023-12-11T19:36:24.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:23 smithi167 ceph-mon[102750]: pgmap v444: 97 pgs: 97 active+clean; 2.4 GiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 206 MiB/s wr, 88 op/s 2023-12-11T19:36:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/1826983622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 msgr2=0x7f9cb81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/1826983622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81043f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009990 tx=0x7f9cb0009ca0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/1826983622 shutdown_connections 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/1826983622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb8107260 0x7f9cb8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/1826983622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81043f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/1826983622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 0x7f9cb8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/1826983622 >> 172.21.15.66:0/1826983622 conn(0x7f9cb80fb990 msgr2=0x7f9cb80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.803+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/1826983622 shutdown_connections 2023-12-11T19:36:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.804+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/1826983622 wait complete. 2023-12-11T19:36:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 Processor -- start 2023-12-11T19:36:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 -- start start 2023-12-11T19:36:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 0x7f9cb8195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb8107260 0x7f9cb819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819cd20 con 0x7f9cb8104930 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819ce60 con 0x7f9cb8107260 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.805+0000 7f9cbc9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819cff0 con 0x7f9cb8102000 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 0x7f9cb8195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:36616/0 (socket says 172.21.15.66:36616) 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 0x7f9cb8195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44612/0 (socket says 172.21.15.66:44612) 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 -- 172.21.15.66:0/2173852522 learned_addr learned my addr 172.21.15.66:0/2173852522 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb8107260 0x7f9cb819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 -- 172.21.15.66:0/2173852522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb8107260 msgr2=0x7f9cb819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb8107260 0x7f9cb819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 -- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 msgr2=0x7f9cb8195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 0x7f9cb8195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.806+0000 7f9cb659c700 1 -- 172.21.15.66:0/2173852522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00095f0 con 0x7f9cb8102000 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.807+0000 7f9cb659c700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81951f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0006010 tx=0x7f9cb0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.807+0000 7f9caf7fe700 1 -- 172.21.15.66:0/2173852522 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb001d070 con 0x7f9cb8102000 2023-12-11T19:36:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.807+0000 7f9caf7fe700 1 -- 172.21.15.66:0/2173852522 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb000bdb0 con 0x7f9cb8102000 2023-12-11T19:36:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.807+0000 7f9caf7fe700 1 -- 172.21.15.66:0/2173852522 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000fda0 con 0x7f9cb8102000 2023-12-11T19:36:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.807+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb819d270 con 0x7f9cb8102000 2023-12-11T19:36:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.807+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb819d690 con 0x7f9cb8102000 2023-12-11T19:36:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.811+0000 7f9caf7fe700 1 -- 172.21.15.66:0/2173852522 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9cb000f460 con 0x7f9cb8102000 2023-12-11T19:36:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.812+0000 7f9caf7fe700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ca40a1500 0x7f9ca40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.813+0000 7f9caf7fe700 1 -- 172.21.15.66:0/2173852522 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cb00cdf80 con 0x7f9cb8102000 2023-12-11T19:36:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.813+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ca40a1500 0x7f9ca40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.813+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ca40a1500 0x7f9ca40a39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0015340 tx=0x7f9ca000f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 Processor -- start 2023-12-11T19:36:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 -- start start 2023-12-11T19:36:24.814 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 0x561da2cde2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cdf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 0x561da2cdfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2c39af0 con 0x561da2cde830 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2c37410 con 0x561da2cdf720 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.814+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2c32d00 con 0x561da2cdded0 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cdf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 0x561da2cdfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cdf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44614/0 (socket says 172.21.15.66:44614) 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb5d9b700 1 -- 172.21.15.66:0/1830793467 learned_addr learned my addr 172.21.15.66:0/1830793467 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 0x561da2cdfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:55234/0 (socket says 172.21.15.66:55234) 2023-12-11T19:36:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1830793467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 msgr2=0x561da2cde2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-11T19:36:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 0x561da2cde2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1830793467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 msgr2=0x561da2cdf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cdf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1830793467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561da2aaf5d0 con 0x561da2cdf720 2023-12-11T19:36:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.815+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 0x561da2cdfba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8011580 tx=0x7f9ca8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73e4fde5ae5c190 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.816+0000 7f9cacff9700 1 -- 172.21.15.66:0/1830793467 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800e040 con 0x561da2cdf720 2023-12-11T19:36:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.816+0000 7f9cacff9700 1 -- 172.21.15.66:0/1830793467 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca800f040 con 0x561da2cdf720 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.816+0000 7f9cacff9700 1 -- 172.21.15.66:0/1830793467 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8004320 con 0x561da2cdf720 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1830793467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 msgr2=0x561da2cdfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 0x561da2cdfba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8011580 tx=0x7f9ca8009920 comp rx=0 tx=0).stop 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1830793467 shutdown_connections 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdf720 0x561da2cdfba0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 0x561da2cde2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1830793467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cdf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1830793467 >> 172.21.15.66:0/1830793467 conn(0x561da2c0d780 msgr2=0x561da2cda880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1830793467 shutdown_connections 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.817+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1830793467 wait complete. 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 -- start start 2023-12-11T19:36:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 Processor -- start 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2ce7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cde830 0x561da2ce8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2bec500 con 0x561da2cdf720 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2bec1a0 con 0x561da2cdded0 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.818+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2ab18c0 con 0x561da2cde830 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cd0238b80 10 client.0 _ll_register_callbacks cb 0x561da2b17a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2ce7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2ce7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:55246/0 (socket says 172.21.15.66:55246) 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 -- 172.21.15.66:0/1989906512 learned_addr learned my addr 172.21.15.66:0/1989906512 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cde830 0x561da2ce8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 -- 172.21.15.66:0/1989906512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cde830 msgr2=0x561da2ce8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cde830 0x561da2ce8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 -- 172.21.15.66:0/1989906512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 msgr2=0x561da2cef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb659c700 1 -- 172.21.15.66:0/1989906512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca8011230 con 0x561da2cdded0 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cde830 0x561da2ce8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.819+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.820+0000 7f9cb659c700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2ce7ec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0097da0 tx=0x7f9cb000bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c49bf0b0a159180 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.820+0000 7f9c9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.820+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1989906512 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb009ba30 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.820+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1989906512 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cb009b330 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.820+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561da2cd1730 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1989906512 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb009f480 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 10 client.26003 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561da2a5bd80 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1989906512 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb00b7020 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1989906512 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9cb00a31a0 con 0x561da2cdded0 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 10 client.26003 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 10 client.26003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 20 client.26003 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2cf3930 0x561da2cf5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 4 client.26003 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.821+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561da2cadee0 con 0x561da2cf3930 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.822+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2cf3930 0x561da2cf5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.822+0000 7f9c9dffb700 10 client.26003 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.822+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2cf3930 0x561da2cf5d50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d2c993d815b93d server_cookie=e1d3732afb120535 in_seq=0 out_seq=0 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.823+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1989906512 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561da2cadee0 con 0x561da2cf3930 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.823+0000 7f9c9dffb700 10 client.26003 handle_command_reply: tid=0 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 client.26003 shutdown 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 msgr2=0x561da2ce7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2ce7ec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0097da0 tx=0x7f9cb000bd20 comp rx=0 tx=0).stop 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 shutdown_connections 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2cf3930 0x561da2cf5d50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2ce7ec0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9c9dffb700 0 client.26003 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1989906512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cde830 0x561da2ce8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.824+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 >> 172.21.15.66:0/1989906512 conn(0x561da2c0d780 msgr2=0x561da2cd8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.825+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 shutdown_connections 2023-12-11T19:36:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.825+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1989906512 wait complete. 2023-12-11T19:36:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.825+0000 7f9cd0238b80 20 client.26003 trim_cache size 0 max 16384 2023-12-11T19:36:24.841 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.841+0000 7f9cd0238b80 1 Processor -- start 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.841+0000 7f9cd0238b80 1 -- start start 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2cefb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdf720 0x561da2cdfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2cef830 con 0x561da2cde830 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2cf05e0 con 0x561da2cdded0 2023-12-11T19:36:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2ce7820 con 0x561da2cdf720 2023-12-11T19:36:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cb5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cb5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44628/0 (socket says 172.21.15.66:44628) 2023-12-11T19:36:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.842+0000 7f9cb5d9b700 1 -- 172.21.15.66:0/247426635 learned_addr learned my addr 172.21.15.66:0/247426635 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb659c700 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2cefb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb5d9b700 1 -- 172.21.15.66:0/247426635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdf720 msgr2=0x561da2cdfb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-11T19:36:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdf720 0x561da2cdfb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb5d9b700 1 -- 172.21.15.66:0/247426635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 msgr2=0x561da2cefb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2cefb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb5d9b700 1 -- 172.21.15.66:0/247426635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561da2a44c00 con 0x561da2cde830 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb659c700 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2cefb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cef1f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0015a90 tx=0x7f9ca001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d40f57752b6a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cad7fa700 1 -- 172.21.15.66:0/247426635 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca002c040 con 0x561da2cde830 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cad7fa700 1 -- 172.21.15.66:0/247426635 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca0013040 con 0x561da2cde830 2023-12-11T19:36:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.843+0000 7f9cad7fa700 1 -- 172.21.15.66:0/247426635 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0012040 con 0x561da2cde830 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 -- 172.21.15.66:0/247426635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 msgr2=0x561da2cef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cef1f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0015a90 tx=0x7f9ca001c000 comp rx=0 tx=0).stop 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 -- 172.21.15.66:0/247426635 shutdown_connections 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cdded0 0x561da2cefb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdf720 0x561da2cdfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/247426635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cde830 0x561da2cef1f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 -- 172.21.15.66:0/247426635 >> 172.21.15.66:0/247426635 conn(0x561da2c0d780 msgr2=0x561da2ce5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 -- 172.21.15.66:0/247426635 shutdown_connections 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.844+0000 7f9cd0238b80 1 -- 172.21.15.66:0/247426635 wait complete. 2023-12-11T19:36:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.845+0000 7f9cd0238b80 1 -- start start 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.845+0000 7f9cd0238b80 1 Processor -- start 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 0x561da2ced080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cde830 0x561da2ced5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2ceeb10 con 0x561da2cdf720 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2ceec80 con 0x561da2cde830 2023-12-11T19:36:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da2ceee40 con 0x561da2cdded0 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cd0238b80 10 client.0 _ll_register_callbacks cb 0x561da2b17a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44640/0 (socket says 172.21.15.66:44640) 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1461004806 learned_addr learned my addr 172.21.15.66:0/1461004806 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb5d9b700 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cde830 0x561da2ced5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1461004806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 msgr2=0x561da2ced080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 0x561da2ced080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cde830 msgr2=0x561da2ced5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.846+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cde830 0x561da2ced5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.847+0000 7f9cb6d9d700 1 -- 172.21.15.66:0/1461004806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca0004240 con 0x561da2cdf720 2023-12-11T19:36:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.847+0000 7f9cb6d9d700 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cee5d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x561da2cdef00 tx=0x7f9ca8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76e8ea918f88132 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.847+0000 7f9c9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.847+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1461004806 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800e040 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.847+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1461004806 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ca8007910 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.847+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ca800f040 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1461004806 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca8007b10 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9cd0238b80 10 client.16302 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561da2a821f0 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1461004806 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca800f040 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1461004806 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ca801b020 con 0x561da2cdf720 2023-12-11T19:36:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9cd0238b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9cd0238b80 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9cd0238b80 20 client.16302 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.848+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2d4adf0 0x561da2d4d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.849+0000 7f9cd0238b80 4 client.16302 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.849+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561da2bf8f70 con 0x561da2d4adf0 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.849+0000 7f9cb659c700 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2d4adf0 0x561da2d4d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.849+0000 7f9cb659c700 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2d4adf0 0x561da2d4d210 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5937656f234122 server_cookie=efef5dbc582c563d in_seq=0 out_seq=0 2023-12-11T19:36:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.849+0000 7f9c9dffb700 10 client.16302 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.863+0000 7f9c9dffb700 1 -- 172.21.15.66:0/1461004806 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561da2bf8f70 con 0x561da2d4adf0 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.863+0000 7f9c9dffb700 10 client.16302 handle_command_reply: tid=0 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.863+0000 7f9cd0238b80 1 client.16302 shutdown 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 msgr2=0x561da2cee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cee5d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x561da2cdef00 tx=0x7f9ca8009510 comp rx=0 tx=0).stop 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 shutdown_connections 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x561da2d4adf0 0x561da2d4d210 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561da2cdf720 0x561da2cee5d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561da2cdded0 0x561da2ced080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 --2- 172.21.15.66:0/1461004806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x561da2cde830 0x561da2ced5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 >> 172.21.15.66:0/1461004806 conn(0x561da2c0d780 msgr2=0x561da2cda460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9c9dffb700 0 client.16302 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 shutdown_connections 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 1 -- 172.21.15.66:0/1461004806 wait complete. 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.864+0000 7f9cd0238b80 20 client.16302 trim_cache size 0 max 16384 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:36:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:36:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ca40a1500 msgr2=0x7f9ca40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ca40a1500 0x7f9ca40a39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0015340 tx=0x7f9ca000f040 comp rx=0 tx=0).stop 2023-12-11T19:36:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 msgr2=0x7f9cb81951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81951f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0006010 tx=0x7f9cb0004730 comp rx=0 tx=0).stop 2023-12-11T19:36:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 shutdown_connections 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9ca40a1500 0x7f9ca40a39c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb8107260 0x7f9cb819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102000 0x7f9cb81951f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 --2- 172.21.15.66:0/2173852522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cb8104930 0x7f9cb8195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 >> 172.21.15.66:0/2173852522 conn(0x7f9cb80fb990 msgr2=0x7f9cb8103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 shutdown_connections 2023-12-11T19:36:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:24.865+0000 7f9cbc9ff700 1 -- 172.21.15.66:0/2173852522 wait complete. 2023-12-11T19:36:26.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:25 smithi066 ceph-mon[97920]: pgmap v445: 97 pgs: 97 active+clean; 3.4 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 290 MiB/s wr, 110 op/s 2023-12-11T19:36:26.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:25 smithi070 ceph-mon[104044]: pgmap v445: 97 pgs: 97 active+clean; 3.4 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 290 MiB/s wr, 110 op/s 2023-12-11T19:36:26.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:25 smithi167 ceph-mon[102750]: pgmap v445: 97 pgs: 97 active+clean; 3.4 GiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 290 MiB/s wr, 110 op/s 2023-12-11T19:36:28.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:27 smithi066 ceph-mon[97920]: pgmap v446: 97 pgs: 97 active+clean; 4.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 346 MiB/s wr, 114 op/s 2023-12-11T19:36:28.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:27 smithi070 ceph-mon[104044]: pgmap v446: 97 pgs: 97 active+clean; 4.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 346 MiB/s wr, 114 op/s 2023-12-11T19:36:28.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:27 smithi167 ceph-mon[102750]: pgmap v446: 97 pgs: 97 active+clean; 4.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 346 MiB/s wr, 114 op/s 2023-12-11T19:36:30.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:29 smithi066 ceph-mon[97920]: pgmap v447: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 346 MiB/s wr, 101 op/s 2023-12-11T19:36:30.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:29 smithi070 ceph-mon[104044]: pgmap v447: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 346 MiB/s wr, 101 op/s 2023-12-11T19:36:30.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:29 smithi167 ceph-mon[102750]: pgmap v447: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 346 MiB/s wr, 101 op/s 2023-12-11T19:36:32.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:31 smithi066 ceph-mon[97920]: pgmap v448: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 383 MiB/s wr, 111 op/s 2023-12-11T19:36:32.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:31 smithi070 ceph-mon[104044]: pgmap v448: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 383 MiB/s wr, 111 op/s 2023-12-11T19:36:32.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:31 smithi167 ceph-mon[102750]: pgmap v448: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 383 MiB/s wr, 111 op/s 2023-12-11T19:36:34.063 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:33 smithi066 ceph-mon[97920]: pgmap v449: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 343 MiB/s wr, 98 op/s 2023-12-11T19:36:34.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:33 smithi070 ceph-mon[104044]: pgmap v449: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 343 MiB/s wr, 98 op/s 2023-12-11T19:36:34.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:33 smithi167 ceph-mon[102750]: pgmap v449: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 343 MiB/s wr, 98 op/s 2023-12-11T19:36:36.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:35 smithi066 ceph-mon[97920]: pgmap v450: 97 pgs: 97 active+clean; 7.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 397 MiB/s wr, 113 op/s 2023-12-11T19:36:36.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:35 smithi070 ceph-mon[104044]: pgmap v450: 97 pgs: 97 active+clean; 7.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 397 MiB/s wr, 113 op/s 2023-12-11T19:36:36.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:35 smithi167 ceph-mon[102750]: pgmap v450: 97 pgs: 97 active+clean; 7.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 397 MiB/s wr, 113 op/s 2023-12-11T19:36:38.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:37 smithi066 ceph-mon[97920]: pgmap v451: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 374 MiB/s wr, 108 op/s 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 -- 172.21.15.66:0/2092550962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc1048d0 msgr2=0x7f9cdc106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/2092550962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc106cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009990 tx=0x7f9ccc009ca0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 -- 172.21.15.66:0/2092550962 shutdown_connections 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/2092550962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/2092550962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cdc107200 0x7f9cdc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/2092550962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc106cc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 -- 172.21.15.66:0/2092550962 >> 172.21.15.66:0/2092550962 conn(0x7f9cdc0fb990 msgr2=0x7f9cdc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 -- 172.21.15.66:0/2092550962 shutdown_connections 2023-12-11T19:36:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.093+0000 7f9ce21ff700 1 -- 172.21.15.66:0/2092550962 wait complete. 2023-12-11T19:36:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.094+0000 7f9ce21ff700 1 Processor -- start 2023-12-11T19:36:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.094+0000 7f9ce21ff700 1 -- start start 2023-12-11T19:36:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.095+0000 7f9ce21ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.095+0000 7f9ce21ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9ce21ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cdc107200 0x7f9cdc1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9ce21ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1a10e0 con 0x7f9cdc101fa0 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9ce21ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1a1220 con 0x7f9cdc1048d0 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9ce21ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1a13b0 con 0x7f9cdc107200 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9cdb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9cdb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42428/0 (socket says 172.21.15.66:42428) 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/1255678293 learned_addr learned my addr 172.21.15.66:0/1255678293 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cdc107200 0x7f9cdc1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.096+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/1255678293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cdc107200 msgr2=0x7f9cdc1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cdc107200 0x7f9cdc1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/1255678293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc1048d0 msgr2=0x7f9cdc199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/1255678293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0095f0 con 0x7f9cdc101fa0 2023-12-11T19:36:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc1995b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c920 tx=0x7f9cc400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.097+0000 7f9cd8ff9700 1 -- 172.21.15.66:0/1255678293 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4007b00 con 0x7f9cdc101fa0 2023-12-11T19:36:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.098+0000 7f9cd8ff9700 1 -- 172.21.15.66:0/1255678293 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cc400ce70 con 0x7f9cdc101fa0 2023-12-11T19:36:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.098+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc1a1690 con 0x7f9cdc101fa0 2023-12-11T19:36:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.098+0000 7f9cd8ff9700 1 -- 172.21.15.66:0/1255678293 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40053e0 con 0x7f9cdc101fa0 2023-12-11T19:36:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.098+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc1a1ba0 con 0x7f9cdc101fa0 2023-12-11T19:36:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 Processor -- start 2023-12-11T19:36:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 -- start start 2023-12-11T19:36:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b27830 0x558950b28140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950a82af0 con 0x558950b27830 2023-12-11T19:36:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950a80410 con 0x558950b28720 2023-12-11T19:36:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.104+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950a7bd00 con 0x558950b26ed0 2023-12-11T19:36:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b272f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:60112/0 (socket says 172.21.15.66:60112) 2023-12-11T19:36:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2573527547 learned_addr learned my addr 172.21.15.66:0/2573527547 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2573527547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 msgr2=0x558950b28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b28ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2573527547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b27830 msgr2=0x558950b28140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b27830 0x558950b28140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.105+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2573527547 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589508f85d0 con 0x558950b26ed0 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.106+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b272f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40098f0 tx=0x7f9cd4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c1089b2f36989 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.106+0000 7f9cd1ffb700 1 -- 172.21.15.66:0/2573527547 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4019070 con 0x558950b26ed0 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.106+0000 7f9cd1ffb700 1 -- 172.21.15.66:0/2573527547 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cd400f040 con 0x558950b26ed0 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.106+0000 7f9cd1ffb700 1 -- 172.21.15.66:0/2573527547 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4007a20 con 0x558950b26ed0 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.107+0000 7f9cd8ff9700 1 -- 172.21.15.66:0/1255678293 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9cc4004750 con 0x7f9cdc101fa0 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.107+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2573527547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 msgr2=0x558950b272f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.107+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b272f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40098f0 tx=0x7f9cd4009c00 comp rx=0 tx=0).stop 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2573527547 shutdown_connections 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b272f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2573527547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b27830 0x558950b28140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2573527547 >> 172.21.15.66:0/2573527547 conn(0x558950a56780 msgr2=0x558950b23880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2573527547 shutdown_connections 2023-12-11T19:36:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.108+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2573527547 wait complete. 2023-12-11T19:36:38.112 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:37 smithi070 ceph-mon[104044]: pgmap v451: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 374 MiB/s wr, 108 op/s 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.109+0000 7f9cf5accb80 1 -- start start 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.109+0000 7f9cf5accb80 1 Processor -- start 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b26ed0 0x558950b396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b39c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b40a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950a35500 con 0x558950b28720 2023-12-11T19:36:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950a351a0 con 0x558950b26ed0 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589508fa8c0 con 0x558950b27830 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cd8ff9700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cc80a14e0 0x7f9cc80a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cf5accb80 10 client.0 _ll_register_callbacks cb 0x558950960a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.111+0000 7f9cd8ff9700 1 -- 172.21.15.66:0/1255678293 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cc40cc4a0 con 0x7f9cdc101fa0 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.112+0000 7f9cdbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b26ed0 0x558950b396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.112+0000 7f9cdbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b26ed0 0x558950b396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:53636/0 (socket says 172.21.15.66:53636) 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.112+0000 7f9cdb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b39c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.112+0000 7f9cdbfff700 1 -- 172.21.15.66:0/84806844 learned_addr learned my addr 172.21.15.66:0/84806844 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.112+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b40a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdaffd700 1 -- 172.21.15.66:0/84806844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 msgr2=0x558950b39c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b39c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdaffd700 1 -- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b26ed0 msgr2=0x558950b396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b26ed0 0x558950b396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdaffd700 1 -- 172.21.15.66:0/84806844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd4003c50 con 0x558950b28720 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b39c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.113+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cc80a14e0 0x7f9cc80a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.114+0000 7f9cdaffd700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b40a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b99563e11765e89 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.114+0000 7f9cbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.115+0000 7f9cbaffd700 1 -- 172.21.15.66:0/84806844 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc01e050 con 0x558950b28720 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.115+0000 7f9cbaffd700 1 -- 172.21.15.66:0/84806844 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ccc005550 con 0x558950b28720 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.115+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cc80a14e0 0x7f9cc80a39a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000f80 tx=0x7f9cd4003bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:38.116 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.115+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558950b1a730 con 0x558950b28720 2023-12-11T19:36:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.116+0000 7f9cbaffd700 1 -- 172.21.15.66:0/84806844 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ccc005710 con 0x558950b28720 2023-12-11T19:36:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.116+0000 7f9cf5accb80 10 client.16311 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.116+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5589508a4d80 con 0x558950b28720 2023-12-11T19:36:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cbaffd700 1 -- 172.21.15.66:0/84806844 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558950b1a730 con 0x558950b28720 2023-12-11T19:36:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cbaffd700 1 -- 172.21.15.66:0/84806844 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9ccc003940 con 0x558950b28720 2023-12-11T19:36:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cf5accb80 10 client.16311 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cf5accb80 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cf5accb80 20 client.16311 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b454b0 0x558950b478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cf5accb80 4 client.16311 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.117+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558950af6ee0 con 0x558950b454b0 2023-12-11T19:36:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.118+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b454b0 0x558950b478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.122+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b454b0 0x558950b478d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ace3fc84a7de51 server_cookie=2d89a5f099b4bb07 in_seq=0 out_seq=0 2023-12-11T19:36:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.122+0000 7f9cbaffd700 10 client.16311 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.123+0000 7f9cbaffd700 1 -- 172.21.15.66:0/84806844 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558950af6ee0 con 0x558950b454b0 2023-12-11T19:36:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.123+0000 7f9cbaffd700 10 client.16311 handle_command_reply: tid=0 2023-12-11T19:36:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.123+0000 7f9cf5accb80 1 client.16311 shutdown 2023-12-11T19:36:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 msgr2=0x558950b40a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b40a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc000c00 tx=0x7f9ccc009ee0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 shutdown_connections 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b454b0 0x558950b478d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b26ed0 0x558950b396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b40a30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/84806844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b39c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 >> 172.21.15.66:0/84806844 conn(0x558950a56780 msgr2=0x558950b21ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cbaffd700 0 client.16311 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 shutdown_connections 2023-12-11T19:36:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.124+0000 7f9cf5accb80 1 -- 172.21.15.66:0/84806844 wait complete. 2023-12-11T19:36:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.125+0000 7f9cf5accb80 20 client.16311 trim_cache size 0 max 16384 2023-12-11T19:36:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 Processor -- start 2023-12-11T19:36:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 -- start start 2023-12-11T19:36:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 0x558950b41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950b41de0 con 0x558950b26ed0 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.144+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950b27670 con 0x558950b28720 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950b41010 con 0x558950b27830 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 0x558950b41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 0x558950b41310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42436/0 (socket says 172.21.15.66:42436) 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/324982981 learned_addr learned my addr 172.21.15.66:0/324982981 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/324982981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 msgr2=0x558950b409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/324982981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 msgr2=0x558950b301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b301b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.145+0000 7f9cdb7fe700 1 -- 172.21.15.66:0/324982981 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55895088dc00 con 0x558950b26ed0 2023-12-11T19:36:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.146+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 0x558950b41310 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4096270 tx=0x7f9cc40b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfa96b70591f58c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.146+0000 7f9cd27fc700 1 -- 172.21.15.66:0/324982981 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc409d950 con 0x558950b26ed0 2023-12-11T19:36:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.146+0000 7f9cd27fc700 1 -- 172.21.15.66:0/324982981 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cc40a1b90 con 0x558950b26ed0 2023-12-11T19:36:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.146+0000 7f9cd27fc700 1 -- 172.21.15.66:0/324982981 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40cc110 con 0x558950b26ed0 2023-12-11T19:36:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 -- 172.21.15.66:0/324982981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 msgr2=0x558950b41310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 0x558950b41310 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4096270 tx=0x7f9cc40b9040 comp rx=0 tx=0).stop 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 -- 172.21.15.66:0/324982981 shutdown_connections 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b28720 0x558950b301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b26ed0 0x558950b41310 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/324982981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b27830 0x558950b409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 -- 172.21.15.66:0/324982981 >> 172.21.15.66:0/324982981 conn(0x558950a56780 msgr2=0x558950b23460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.147+0000 7f9cf5accb80 1 -- 172.21.15.66:0/324982981 shutdown_connections 2023-12-11T19:36:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.148+0000 7f9cf5accb80 1 -- 172.21.15.66:0/324982981 wait complete. 2023-12-11T19:36:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.149+0000 7f9cf5accb80 1 -- start start 2023-12-11T19:36:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.149+0000 7f9cf5accb80 1 Processor -- start 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.150+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b27830 0x558950b3e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cf5accb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b3f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950b3fb20 con 0x558950b28720 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950b3fc90 con 0x558950b27830 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cf5accb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558950b3fe50 con 0x558950b26ed0 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cf5accb80 10 client.0 _ll_register_callbacks cb 0x7f9cd4007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cdbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b3f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cdbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b3f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42440/0 (socket says 172.21.15.66:42440) 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2292154024 learned_addr learned my addr 172.21.15.66:0/2292154024 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.151+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2292154024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 msgr2=0x558950b3e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b3e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b27830 msgr2=0x558950b3e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b27830 0x558950b3e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdbfff700 1 -- 172.21.15.66:0/2292154024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc1a1ba0 con 0x558950b28720 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cdbfff700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b3f570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40049d0 tx=0x7f9cd4009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b618826badd5bf32 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.152+0000 7f9cbaffd700 1 -- 172.21.15.66:0/2292154024 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4032070 con 0x558950b28720 2023-12-11T19:36:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cbaffd700 1 -- 172.21.15.66:0/2292154024 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cd4015aa0 con 0x558950b28720 2023-12-11T19:36:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cd400f040 con 0x558950b28720 2023-12-11T19:36:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cbaffd700 1 -- 172.21.15.66:0/2292154024 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cd4019040 con 0x558950b28720 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 10 client.16317 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5589508cb1f0 con 0x558950b28720 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cbaffd700 1 -- 172.21.15.66:0/2292154024 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cd400f040 con 0x558950b28720 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cbaffd700 1 -- 172.21.15.66:0/2292154024 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9cd4035020 con 0x558950b28720 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 10 client.16317 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 20 client.16317 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b9c3a0 0x558950b9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 4 client.16317 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:36:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.153+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558950a41f70 con 0x558950b9c3a0 2023-12-11T19:36:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.156+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b9c3a0 0x558950b9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.156+0000 7f9cbaffd700 10 client.16317 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.156+0000 7f9cdb7fe700 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b9c3a0 0x558950b9e7c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29b22298355e0e4 server_cookie=a737006efb21a5cc in_seq=0 out_seq=0 2023-12-11T19:36:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.171+0000 7f9cbaffd700 1 -- 172.21.15.66:0/2292154024 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558950a41f70 con 0x558950b9c3a0 2023-12-11T19:36:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.171+0000 7f9cbaffd700 10 client.16317 handle_command_reply: tid=0 2023-12-11T19:36:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.171+0000 7f9cf5accb80 1 client.16317 shutdown 2023-12-11T19:36:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.171+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 msgr2=0x558950b3f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.171+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b3f570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40049d0 tx=0x7f9cd4009100 comp rx=0 tx=0).stop 2023-12-11T19:36:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 shutdown_connections 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x558950b9c3a0 0x558950b9e7c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558950b28720 0x558950b3f570 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558950b26ed0 0x558950b3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 --2- 172.21.15.66:0/2292154024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x558950b27830 0x558950b3e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cbaffd700 0 client.16317 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 >> 172.21.15.66:0/2292154024 conn(0x558950a56780 msgr2=0x558950b2b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 shutdown_connections 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 1 -- 172.21.15.66:0/2292154024 wait complete. 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.172+0000 7f9cf5accb80 20 client.16317 trim_cache size 0 max 16384 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:36:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:36:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:36:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:36:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cc80a14e0 msgr2=0x7f9cc80a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cc80a14e0 0x7f9cc80a39a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000f80 tx=0x7f9cd4003bb0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 msgr2=0x7f9cdc1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc1995b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c920 tx=0x7f9cc400cc30 comp rx=0 tx=0).stop 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 shutdown_connections 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f9cc80a14e0 0x7f9cc80a39a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cdc107200 0x7f9cdc1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cdc101fa0 0x7f9cdc1995b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 --2- 172.21.15.66:0/1255678293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cdc1048d0 0x7f9cdc199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 >> 172.21.15.66:0/1255678293 conn(0x7f9cdc0fb990 msgr2=0x7f9cdc0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 shutdown_connections 2023-12-11T19:36:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:38.174+0000 7f9ce21ff700 1 -- 172.21.15.66:0/1255678293 wait complete. 2023-12-11T19:36:38.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:37 smithi167 ceph-mon[102750]: pgmap v451: 97 pgs: 97 active+clean; 7.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 374 MiB/s wr, 108 op/s 2023-12-11T19:36:40.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:39 smithi066 ceph-mon[97920]: pgmap v452: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 349 MiB/s wr, 98 op/s 2023-12-11T19:36:40.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:39 smithi070 ceph-mon[104044]: pgmap v452: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 349 MiB/s wr, 98 op/s 2023-12-11T19:36:40.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:39 smithi167 ceph-mon[102750]: pgmap v452: 97 pgs: 97 active+clean; 8.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 349 MiB/s wr, 98 op/s 2023-12-11T19:36:40.490 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:36:40.491 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 initial writers = 376110.59 kB/sec 2023-12-11T19:36:40.491 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 initial writers = 376099.32 kB/sec 2023-12-11T19:36:40.491 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 376110.59 kB/sec 2023-12-11T19:36:40.491 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 376110.59 kB/sec 2023-12-11T19:36:40.491 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 376110.59 kB/sec 2023-12-11T19:36:40.491 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 10485760.00 kB 2023-12-11T19:36:42.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:41 smithi070 ceph-mon[104044]: pgmap v453: 97 pgs: 97 active+clean; 9.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 385 MiB/s wr, 111 op/s 2023-12-11T19:36:42.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:41 smithi167 ceph-mon[102750]: pgmap v453: 97 pgs: 97 active+clean; 9.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 385 MiB/s wr, 111 op/s 2023-12-11T19:36:42.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:41 smithi066 ceph-mon[97920]: pgmap v453: 97 pgs: 97 active+clean; 9.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 385 MiB/s wr, 111 op/s 2023-12-11T19:36:44.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:43 smithi070 ceph-mon[104044]: pgmap v454: 97 pgs: 97 active+clean; 9.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 339 MiB/s wr, 96 op/s 2023-12-11T19:36:44.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:43 smithi167 ceph-mon[102750]: pgmap v454: 97 pgs: 97 active+clean; 9.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 339 MiB/s wr, 96 op/s 2023-12-11T19:36:44.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:43 smithi066 ceph-mon[97920]: pgmap v454: 97 pgs: 97 active+clean; 9.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 339 MiB/s wr, 96 op/s 2023-12-11T19:36:46.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:45 smithi070 ceph-mon[104044]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 343 MiB/s wr, 98 op/s 2023-12-11T19:36:46.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:45 smithi167 ceph-mon[102750]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 343 MiB/s wr, 98 op/s 2023-12-11T19:36:46.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:45 smithi066 ceph-mon[97920]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 343 MiB/s wr, 98 op/s 2023-12-11T19:36:48.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:47 smithi070 ceph-mon[104044]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 294 MiB/s wr, 87 op/s 2023-12-11T19:36:48.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:47 smithi167 ceph-mon[102750]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 294 MiB/s wr, 87 op/s 2023-12-11T19:36:48.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:47 smithi066 ceph-mon[97920]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 294 MiB/s wr, 87 op/s 2023-12-11T19:36:50.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:49 smithi070 ceph-mon[104044]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 280 MiB/s wr, 81 op/s 2023-12-11T19:36:50.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:49 smithi167 ceph-mon[102750]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 280 MiB/s wr, 81 op/s 2023-12-11T19:36:50.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:49 smithi066 ceph-mon[97920]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 280 MiB/s wr, 81 op/s 2023-12-11T19:36:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.411+0000 7f51eabff700 1 -- 172.21.15.66:0/1803462871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101620 msgr2=0x7f51e4101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.411+0000 7f51eabff700 1 --2- 172.21.15.66:0/1803462871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101620 0x7f51e4101a80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f51d00099c0 tx=0x7f51d0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 -- 172.21.15.66:0/1803462871 shutdown_connections 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 --2- 172.21.15.66:0/1803462871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e4100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 --2- 172.21.15.66:0/1803462871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101fc0 0x7f51e4108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 --2- 172.21.15.66:0/1803462871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101620 0x7f51e4101a80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 -- 172.21.15.66:0/1803462871 >> 172.21.15.66:0/1803462871 conn(0x7f51e40fb9c0 msgr2=0x7f51e40fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 -- 172.21.15.66:0/1803462871 shutdown_connections 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.412+0000 7f51eabff700 1 -- 172.21.15.66:0/1803462871 wait complete. 2023-12-11T19:36:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.413+0000 7f51eabff700 1 Processor -- start 2023-12-11T19:36:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.413+0000 7f51eabff700 1 -- start start 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.413+0000 7f51eabff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.413+0000 7f51eabff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101620 0x7f51e4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51eabff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101fc0 0x7f51e41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51eabff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41a1120 con 0x7f51e4101fc0 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51eabff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41a1260 con 0x7f51e4100430 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51eabff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41a13f0 con 0x7f51e4101620 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51e899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51e919c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101fc0 0x7f51e41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51e899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e41995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:42646/0 (socket says 172.21.15.66:42646) 2023-12-11T19:36:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.414+0000 7f51e899b700 1 -- 172.21.15.66:0/327880863 learned_addr learned my addr 172.21.15.66:0/327880863 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e919c700 1 -- 172.21.15.66:0/327880863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101620 msgr2=0x7f51e4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e3fff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101620 0x7f51e4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e919c700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101620 0x7f51e4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e919c700 1 -- 172.21.15.66:0/327880863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 msgr2=0x7f51e41995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e919c700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e41995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e919c700 1 -- 172.21.15.66:0/327880863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00095f0 con 0x7f51e4101fc0 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e3fff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101620 0x7f51e4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e899b700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e919c700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101fc0 0x7f51e41a0b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00efc0 tx=0x7f51dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e1ffb700 1 -- 172.21.15.66:0/327880863 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc018070 con 0x7f51e4101fc0 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e41a1670 con 0x7f51e4101fc0 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e1ffb700 1 -- 172.21.15.66:0/327880863 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51dc00f040 con 0x7f51e4101fc0 2023-12-11T19:36:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.415+0000 7f51e1ffb700 1 -- 172.21.15.66:0/327880863 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc003cf0 con 0x7f51e4101fc0 2023-12-11T19:36:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.416+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e41a1b30 con 0x7f51e4101fc0 2023-12-11T19:36:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.420+0000 7f51fe5b0b80 1 Processor -- start 2023-12-11T19:36:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.421+0000 7f51e1ffb700 1 -- 172.21.15.66:0/327880863 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f51dc00ca50 con 0x7f51e4101fc0 2023-12-11T19:36:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.421+0000 7f51fe5b0b80 1 -- start start 2023-12-11T19:36:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.421+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.421+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.421+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978ba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.421+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56399780baf0 con 0x5639978b9f30 2023-12-11T19:36:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563997809410 con 0x5639978b9040 2023-12-11T19:36:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563997804d00 con 0x5639978b86e0 2023-12-11T19:36:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978ba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978ba3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39554/0 (socket says 172.21.15.66:39554) 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51e3fff700 1 -- 172.21.15.66:0/3762828306 learned_addr learned my addr 172.21.15.66:0/3762828306 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51e3fff700 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.422+0000 7f51e1ffb700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f51d40a1480 0x7f51d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e3fff700 1 -- 172.21.15.66:0/3762828306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 msgr2=0x5639978b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e3fff700 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e3fff700 1 -- 172.21.15.66:0/3762828306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 msgr2=0x5639978b9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e3fff700 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e3fff700 1 -- 172.21.15.66:0/3762828306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639976815d0 con 0x5639978b9f30 2023-12-11T19:36:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e1ffb700 1 -- 172.21.15.66:0/327880863 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51dc0cc970 con 0x7f51e4101fc0 2023-12-11T19:36:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e3fff700 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978ba3b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f51d00036c0 tx=0x7f51d0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558ff518e60a0297 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51e919c700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f51d40a1480 0x7f51d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51ceffd700 1 -- 172.21.15.66:0/3762828306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000b950 con 0x5639978b9f30 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.423+0000 7f51ceffd700 1 -- 172.21.15.66:0/3762828306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51d001c070 con 0x5639978b9f30 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.424+0000 7f51ceffd700 1 -- 172.21.15.66:0/3762828306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00039c0 con 0x5639978b9f30 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.424+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/3762828306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 msgr2=0x5639978ba3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.424+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978ba3b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f51d00036c0 tx=0x7f51d0005fb0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.424+0000 7f51e919c700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f51d40a1480 0x7f51d40a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ef90 tx=0x7f51dc021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/3762828306 shutdown_connections 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978ba3b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/3762828306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/3762828306 >> 172.21.15.66:0/3762828306 conn(0x5639977df780 msgr2=0x5639978ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/3762828306 shutdown_connections 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.425+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/3762828306 wait complete. 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.426+0000 7f51fe5b0b80 1 -- start start 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.426+0000 7f51fe5b0b80 1 Processor -- start 2023-12-11T19:36:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.426+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 0x5639978c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639977be500 con 0x5639978b9040 2023-12-11T19:36:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639977be1a0 con 0x5639978b9f30 2023-12-11T19:36:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639976838c0 con 0x5639978b86e0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51fe5b0b80 10 client.0 _ll_register_callbacks cb 0x5639976e9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51e899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 0x5639978c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51e899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 0x5639978c2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39560/0 (socket says 172.21.15.66:39560) 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51e899b700 1 -- 172.21.15.66:0/2642443902 learned_addr learned my addr 172.21.15.66:0/2642443902 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.427+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.428+0000 7f51e899b700 1 -- 172.21.15.66:0/2642443902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 msgr2=0x5639978c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.428+0000 7f51e899b700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.428+0000 7f51e899b700 1 -- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 msgr2=0x5639978c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.428+0000 7f51e899b700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.428+0000 7f51e899b700 1 -- 172.21.15.66:0/2642443902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e41091c0 con 0x5639978b9040 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.428+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51e899b700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 0x5639978c2c30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f51d8011660 tx=0x7f51d8011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd272fbc5acf288 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:36:51.429 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51c3fff700 1 -- 172.21.15.66:0/2642443902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8009760 con 0x5639978b9040 2023-12-11T19:36:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51c3fff700 1 -- 172.21.15.66:0/2642443902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51d8004a30 con 0x5639978b9040 2023-12-11T19:36:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639978a3730 con 0x5639978b9040 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51c3fff700 1 -- 172.21.15.66:0/2642443902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d8004bf0 con 0x5639978b9040 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51fe5b0b80 10 client.16329 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.429+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56399762dd80 con 0x5639978b9040 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51c3fff700 1 -- 172.21.15.66:0/2642443902 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5639978a3730 con 0x5639978b9040 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51c3fff700 1 -- 172.21.15.66:0/2642443902 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51d80128c0 con 0x5639978b9040 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51fe5b0b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51fe5b0b80 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51fe5b0b80 20 client.16329 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639978ce580 0x5639978d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51fe5b0b80 4 client.16329 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56399787fee0 con 0x5639978ce580 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.430+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639978ce580 0x5639978d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.431+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639978ce580 0x5639978d09a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48e8daf23b847fb server_cookie=227ad841542c6c74 in_seq=0 out_seq=0 2023-12-11T19:36:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.431+0000 7f51c3fff700 10 client.16329 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.432+0000 7f51c3fff700 1 -- 172.21.15.66:0/2642443902 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56399787fee0 con 0x5639978ce580 2023-12-11T19:36:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.432+0000 7f51c3fff700 10 client.16329 handle_command_reply: tid=0 2023-12-11T19:36:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.432+0000 7f51fe5b0b80 1 client.16329 shutdown 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 msgr2=0x5639978c2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 0x5639978c2c30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f51d8011660 tx=0x7f51d8011970 comp rx=0 tx=0).stop 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 shutdown_connections 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639978ce580 0x5639978d09a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51c3fff700 0 client.16329 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2642443902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9040 0x5639978c2c30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 >> 172.21.15.66:0/2642443902 conn(0x5639977df780 msgr2=0x5639978aaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 shutdown_connections 2023-12-11T19:36:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2642443902 wait complete. 2023-12-11T19:36:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.433+0000 7f51fe5b0b80 20 client.16329 trim_cache size 0 max 16384 2023-12-11T19:36:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.454+0000 7f51fe5b0b80 1 Processor -- start 2023-12-11T19:36:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.454+0000 7f51fe5b0b80 1 -- start start 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.454+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 0x5639978d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b9040 0x5639978c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639978cb9f0 con 0x5639978b86e0 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639978ce310 con 0x5639978b9f30 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639978c0ec0 con 0x5639978b9040 2023-12-11T19:36:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 0x5639978d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 0x5639978d52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39562/0 (socket says 172.21.15.66:39562) 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51e3fff700 1 -- 172.21.15.66:0/1377293497 learned_addr learned my addr 172.21.15.66:0/1377293497 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.455+0000 7f51e899b700 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b9040 0x5639978c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51e3fff700 1 -- 172.21.15.66:0/1377293497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b9040 msgr2=0x5639978c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51e3fff700 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b9040 0x5639978c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51e3fff700 1 -- 172.21.15.66:0/1377293497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 msgr2=0x5639978c26f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51e3fff700 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c26f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51e3fff700 1 -- 172.21.15.66:0/1377293497 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563997616c00 con 0x5639978b86e0 2023-12-11T19:36:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51e3fff700 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 0x5639978d52c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f51d00036c0 tx=0x7f51d0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c3fc1eccacccf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.456+0000 7f51cf7fe700 1 -- 172.21.15.66:0/1377293497 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d002c040 con 0x5639978b86e0 2023-12-11T19:36:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51cf7fe700 1 -- 172.21.15.66:0/1377293497 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51d000bc70 con 0x5639978b86e0 2023-12-11T19:36:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51cf7fe700 1 -- 172.21.15.66:0/1377293497 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0003ba0 con 0x5639978b86e0 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/1377293497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 msgr2=0x5639978d52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 0x5639978d52c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f51d00036c0 tx=0x7f51d0009cd0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/1377293497 shutdown_connections 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9f30 0x5639978c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b86e0 0x5639978d52c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/1377293497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b9040 0x5639978c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.457+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/1377293497 >> 172.21.15.66:0/1377293497 conn(0x5639977df780 msgr2=0x5639978c19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.458+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/1377293497 shutdown_connections 2023-12-11T19:36:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.458+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/1377293497 wait complete. 2023-12-11T19:36:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.458+0000 7f51fe5b0b80 1 -- start start 2023-12-11T19:36:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.458+0000 7f51fe5b0b80 1 Processor -- start 2023-12-11T19:36:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639978c8c60 con 0x5639978b9f30 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639978c8dd0 con 0x5639978b9040 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639978c8f90 con 0x5639978b86e0 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51fe5b0b80 10 client.0 _ll_register_callbacks cb 0x5639976e9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51e899b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.459+0000 7f51e3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:43506/0 (socket says 172.21.15.66:43506) 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e3fff700 1 -- 172.21.15.66:0/2772386497 learned_addr learned my addr 172.21.15.66:0/2772386497 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e899b700 1 -- 172.21.15.66:0/2772386497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 msgr2=0x5639978c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e899b700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e899b700 1 -- 172.21.15.66:0/2772386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 msgr2=0x5639978c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e899b700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e899b700 1 -- 172.21.15.66:0/2772386497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d002f040 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51e899b700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978c7710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005950 tx=0x7f51d8012910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba87e73b76a478e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.460+0000 7f51c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.461+0000 7f51c3fff700 1 -- 172.21.15.66:0/2772386497 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8013ea0 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.461+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51d001c070 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.461+0000 7f51c3fff700 1 -- 172.21.15.66:0/2772386497 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51d8020030 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.461+0000 7f51c3fff700 1 -- 172.21.15.66:0/2772386497 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d80052d0 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.461+0000 7f51fe5b0b80 10 client.26027 fetch_fsmap learned FSMap version 30 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.461+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5639976541f0 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51c3fff700 1 -- 172.21.15.66:0/2772386497 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51d001c070 con 0x5639978b9040 2023-12-11T19:36:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51c3fff700 1 -- 172.21.15.66:0/2772386497 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51d8003cd0 con 0x5639978b9040 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51fe5b0b80 10 client.26027 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51fe5b0b80 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51fe5b0b80 20 client.26027 populate_metadata read hostname 'smithi066' 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639979253c0 0x5639979277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51fe5b0b80 4 client.26027 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639977caf70 con 0x5639979253c0 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.462+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639979253c0 0x5639979277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.464+0000 7f51e3fff700 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639979253c0 0x5639979277e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4307c0b1d7d9d329 server_cookie=51f9a6d9f28d23b3 in_seq=0 out_seq=0 2023-12-11T19:36:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.464+0000 7f51c3fff700 10 client.26027 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.473+0000 7f51c3fff700 1 -- 172.21.15.66:0/2772386497 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639977caf70 con 0x5639979253c0 2023-12-11T19:36:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.473+0000 7f51c3fff700 10 client.26027 handle_command_reply: tid=0 2023-12-11T19:36:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.473+0000 7f51fe5b0b80 1 client.26027 shutdown 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 msgr2=0x5639978c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978c7710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005950 tx=0x7f51d8012910 comp rx=0 tx=0).stop 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 shutdown_connections 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x5639979253c0 0x5639979277e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5639978b9f30 0x5639978c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5639978b86e0 0x5639978c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 --2- 172.21.15.66:0/2772386497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5639978b9040 0x5639978c7710 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005950 tx=0x7f51d8012910 comp rx=0 tx=0).stop 2023-12-11T19:36:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51c3fff700 0 client.26027 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:36:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.474+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 >> 172.21.15.66:0/2772386497 conn(0x5639977df780 msgr2=0x5639978c24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.475+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 shutdown_connections 2023-12-11T19:36:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.475+0000 7f51fe5b0b80 1 -- 172.21.15.66:0/2772386497 wait complete. 2023-12-11T19:36:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.476+0000 7f51fe5b0b80 20 client.26027 trim_cache size 0 max 16384 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.477+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f51d40a1480 msgr2=0x7f51d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.477+0000 7f51eabff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f51d40a1480 0x7f51d40a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ef90 tx=0x7f51dc021000 comp rx=0 tx=0).stop 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.477+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101fc0 msgr2=0x7f51e41a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:36:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.477+0000 7f51eabff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101fc0 0x7f51e41a0b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00efc0 tx=0x7f51dc00c3a0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 shutdown_connections 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f51d40a1480 0x7f51d40a3940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51e4100430 0x7f51e41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51e4101fc0 0x7f51e41a0b50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 --2- 172.21.15.66:0/327880863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51e4101620 0x7f51e4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 >> 172.21.15.66:0/327880863 conn(0x7f51e40fb9c0 msgr2=0x7f51e40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:36:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 shutdown_connections 2023-12-11T19:36:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:36:51.478+0000 7f51eabff700 1 -- 172.21.15.66:0/327880863 wait complete. 2023-12-11T19:36:52.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:51 smithi070 ceph-mon[104044]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 348 MiB/s wr, 101 op/s 2023-12-11T19:36:52.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:51 smithi167 ceph-mon[102750]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 348 MiB/s wr, 101 op/s 2023-12-11T19:36:52.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:51 smithi066 ceph-mon[97920]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 348 MiB/s wr, 101 op/s 2023-12-11T19:36:54.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:53 smithi070 ceph-mon[104044]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 330 MiB/s wr, 92 op/s 2023-12-11T19:36:54.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:53 smithi167 ceph-mon[102750]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 330 MiB/s wr, 92 op/s 2023-12-11T19:36:54.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:53 smithi066 ceph-mon[97920]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 330 MiB/s wr, 92 op/s 2023-12-11T19:36:56.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:55 smithi070 ceph-mon[104044]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 419 MiB/s wr, 116 op/s 2023-12-11T19:36:56.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:55 smithi167 ceph-mon[102750]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 419 MiB/s wr, 116 op/s 2023-12-11T19:36:56.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:55 smithi066 ceph-mon[97920]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 419 MiB/s wr, 116 op/s 2023-12-11T19:36:58.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:57 smithi070 ceph-mon[104044]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 465 MiB/s wr, 129 op/s 2023-12-11T19:36:58.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:57 smithi167 ceph-mon[102750]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 465 MiB/s wr, 129 op/s 2023-12-11T19:36:58.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:57 smithi066 ceph-mon[97920]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 465 MiB/s wr, 129 op/s 2023-12-11T19:37:00.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:36:59 smithi070 ceph-mon[104044]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 482 MiB/s wr, 131 op/s 2023-12-11T19:37:00.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:36:59 smithi167 ceph-mon[102750]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 482 MiB/s wr, 131 op/s 2023-12-11T19:37:00.313 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:36:59 smithi066 ceph-mon[97920]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 482 MiB/s wr, 131 op/s 2023-12-11T19:37:02.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:01 smithi167 ceph-mon[102750]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 542 MiB/s wr, 149 op/s 2023-12-11T19:37:02.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:01 smithi066 ceph-mon[97920]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 542 MiB/s wr, 149 op/s 2023-12-11T19:37:02.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:01 smithi070 ceph-mon[104044]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 542 MiB/s wr, 149 op/s 2023-12-11T19:37:03.269 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:37:03.270 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 rewriters = 528023.44 kB/sec 2023-12-11T19:37:03.270 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 rewriters = 527998.18 kB/sec 2023-12-11T19:37:03.270 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 528023.44 kB/sec 2023-12-11T19:37:03.270 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 528023.44 kB/sec 2023-12-11T19:37:03.270 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 528023.44 kB/sec 2023-12-11T19:37:03.270 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 10485760.00 kB 2023-12-11T19:37:04.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:03 smithi167 ceph-mon[102750]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 480 MiB/s wr, 131 op/s 2023-12-11T19:37:04.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:03 smithi066 ceph-mon[97920]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 480 MiB/s wr, 131 op/s 2023-12-11T19:37:04.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:03 smithi070 ceph-mon[104044]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 480 MiB/s wr, 131 op/s 2023-12-11T19:37:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 -- 172.21.15.66:0/3210904111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 msgr2=0x7f8914101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 --2- 172.21.15.66:0/3210904111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f8914101a70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f89080099c0 tx=0x7f8908009cd0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 -- 172.21.15.66:0/3210904111 shutdown_connections 2023-12-11T19:37:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 --2- 172.21.15.66:0/3210904111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 0x7f8914100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 --2- 172.21.15.66:0/3210904111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8914101fb0 0x7f8914108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 --2- 172.21.15.66:0/3210904111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f8914101a70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 -- 172.21.15.66:0/3210904111 >> 172.21.15.66:0/3210904111 conn(0x7f89140fb990 msgr2=0x7f89140fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 -- 172.21.15.66:0/3210904111 shutdown_connections 2023-12-11T19:37:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.648+0000 7f89197ff700 1 -- 172.21.15.66:0/3210904111 wait complete. 2023-12-11T19:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.649+0000 7f89197ff700 1 Processor -- start 2023-12-11T19:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.649+0000 7f89197ff700 1 -- start start 2023-12-11T19:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89197ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 0x7f8914104360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89197ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f89141029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89197ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8914101fb0 0x7f8914102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89197ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89141036d0 con 0x7f8914101610 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89197ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914103840 con 0x7f8914100420 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89197ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89141a1480 con 0x7f8914101fb0 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89127fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f89141029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89127fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f89141029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52452/0 (socket says 172.21.15.66:52452) 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89127fc700 1 -- 172.21.15.66:0/38339445 learned_addr learned my addr 172.21.15.66:0/38339445 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f8912ffd700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 0x7f8914104360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.650+0000 7f89137fe700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8914101fb0 0x7f8914102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89127fc700 1 -- 172.21.15.66:0/38339445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8914101fb0 msgr2=0x7f8914102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89127fc700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8914101fb0 0x7f8914102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89127fc700 1 -- 172.21.15.66:0/38339445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 msgr2=0x7f8914104360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89127fc700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 0x7f8914104360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89127fc700 1 -- 172.21.15.66:0/38339445 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080095f0 con 0x7f8914101610 2023-12-11T19:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f8912ffd700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 0x7f8914104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89127fc700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f89141029f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8908009990 tx=0x7f890800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f88fbfff700 1 -- 172.21.15.66:0/38339445 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890801d070 con 0x7f8914101610 2023-12-11T19:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f88fbfff700 1 -- 172.21.15.66:0/38339445 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f890800bd10 con 0x7f8914101610 2023-12-11T19:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89141a16b0 con 0x7f8914101610 2023-12-11T19:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f88fbfff700 1 -- 172.21.15.66:0/38339445 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908017650 con 0x7f8914101610 2023-12-11T19:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.651+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89141a1ad0 con 0x7f8914101610 2023-12-11T19:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.654+0000 7f88fbfff700 1 -- 172.21.15.66:0/38339445 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8908022470 con 0x7f8914101610 2023-12-11T19:37:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.656+0000 7f892d0f8b80 1 Processor -- start 2023-12-11T19:37:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 -- start start 2023-12-11T19:37:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f88fbfff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88fc0a1500 0x7f88fc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c9662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c967140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c967ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c8c1af0 con 0x56173c967720 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c8bf410 con 0x56173c965ed0 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c8bad00 con 0x56173c966830 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.657+0000 7f88fbfff700 1 -- 172.21.15.66:0/38339445 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f89080ce050 con 0x7f8914101610 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c9662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f89137fe700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88fc0a1500 0x7f88fc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c9662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:45144/0 (socket says 172.21.15.66:45144) 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 -- 172.21.15.66:0/4017224575 learned_addr learned my addr 172.21.15.66:0/4017224575 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f89127fc700 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c967140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f89137fe700 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c967ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 -- 172.21.15.66:0/4017224575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 msgr2=0x56173c967140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c967140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 -- 172.21.15.66:0/4017224575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 msgr2=0x56173c967ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c967ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 -- 172.21.15.66:0/4017224575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173c7375d0 con 0x56173c965ed0 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f8912ffd700 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c9662f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f890400bd40 tx=0x7f890400f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0d7c702c96ab32 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.658+0000 7f89137fe700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88fc0a1500 0x7f88fc0a39c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56173c966f00 tx=0x7f890c009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.659+0000 7f88f97fa700 1 -- 172.21.15.66:0/4017224575 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400fda0 con 0x56173c965ed0 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.659+0000 7f88f97fa700 1 -- 172.21.15.66:0/4017224575 <== mon.2 v2:172.21.15.70:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f890400be90 con 0x56173c965ed0 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.659+0000 7f88f97fa700 1 -- 172.21.15.66:0/4017224575 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890401b670 con 0x56173c965ed0 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.659+0000 7f892d0f8b80 1 -- 172.21.15.66:0/4017224575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 msgr2=0x56173c9662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.659+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c9662f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f890400bd40 tx=0x7f890400f640 comp rx=0 tx=0).stop 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 -- 172.21.15.66:0/4017224575 shutdown_connections 2023-12-11T19:37:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c9662f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c967ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/4017224575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c967140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 -- 172.21.15.66:0/4017224575 >> 172.21.15.66:0/4017224575 conn(0x56173c895780 msgr2=0x56173c962880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 -- 172.21.15.66:0/4017224575 shutdown_connections 2023-12-11T19:37:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 -- 172.21.15.66:0/4017224575 wait complete. 2023-12-11T19:37:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 -- start start 2023-12-11T19:37:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.660+0000 7f892d0f8b80 1 Processor -- start 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c965ed0 0x56173c96ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 0x56173c970490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c977220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c874500 con 0x56173c967720 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c8741a0 con 0x56173c966830 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c7398c0 con 0x56173c965ed0 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f892d0f8b80 10 client.0 _ll_register_callbacks cb 0x56173c79fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f89137fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c977220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f89127fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 0x56173c970490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f89127fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 0x56173c970490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.66:45150/0 (socket says 172.21.15.66:45150) 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f89127fc700 1 -- 172.21.15.66:0/1404009368 learned_addr learned my addr 172.21.15.66:0/1404009368 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f89137fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c977220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52464/0 (socket says 172.21.15.66:52464) 2023-12-11T19:37:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.661+0000 7f8912ffd700 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c965ed0 0x56173c96ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f89127fc700 1 -- 172.21.15.66:0/1404009368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c965ed0 msgr2=0x56173c96ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f89127fc700 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c965ed0 0x56173c96ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f89127fc700 1 -- 172.21.15.66:0/1404009368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 msgr2=0x56173c977220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f89127fc700 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c977220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f89127fc700 1 -- 172.21.15.66:0/1404009368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890400b9a0 con 0x56173c966830 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f89127fc700 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 0x56173c970490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8908005cb0 tx=0x7f890809b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5e061f544ff6728 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f88f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f88f6ff5700 1 -- 172.21.15.66:0/1404009368 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080144a0 con 0x56173c966830 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f88f6ff5700 1 -- 172.21.15.66:0/1404009368 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f890809bac0 con 0x56173c966830 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.662+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56173c959730 con 0x56173c966830 2023-12-11T19:37:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f88f6ff5700 1 -- 172.21.15.66:0/1404009368 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8908015300 con 0x56173c966830 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 10 client.26039 fetch_fsmap learned FSMap version 30 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56173c6e3d80 con 0x56173c966830 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f88f6ff5700 1 -- 172.21.15.66:0/1404009368 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89080b3020 con 0x56173c966830 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f88f6ff5700 1 -- 172.21.15.66:0/1404009368 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f890809f170 con 0x56173c966830 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 10 client.26039 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 20 client.26039 populate_metadata read hostname 'smithi066' 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c97b9b0 0x56173c97ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 4 client.26039 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:37:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.663+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56173c935ee0 con 0x56173c97b9b0 2023-12-11T19:37:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.664+0000 7f8912ffd700 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c97b9b0 0x56173c97ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.664+0000 7f8912ffd700 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c97b9b0 0x56173c97ddd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5228cbbca00db4dc server_cookie=3def1f3ec41d080a in_seq=0 out_seq=0 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.664+0000 7f88f6ff5700 10 client.26039 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.665+0000 7f88f6ff5700 1 -- 172.21.15.66:0/1404009368 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56173c935ee0 con 0x56173c97b9b0 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.665+0000 7f88f6ff5700 10 client.26039 handle_command_reply: tid=0 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.665+0000 7f892d0f8b80 1 client.26039 shutdown 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.665+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 msgr2=0x56173c970490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.665+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 0x56173c970490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8908005cb0 tx=0x7f890809b260 comp rx=0 tx=0).stop 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 shutdown_connections 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c97b9b0 0x56173c97ddd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c967720 0x56173c977220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c965ed0 0x56173c96ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1404009368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c966830 0x56173c970490 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f88f6ff5700 0 client.26039 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 >> 172.21.15.66:0/1404009368 conn(0x56173c895780 msgr2=0x56173c960ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 shutdown_connections 2023-12-11T19:37:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1404009368 wait complete. 2023-12-11T19:37:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.666+0000 7f892d0f8b80 20 client.26039 trim_cache size 0 max 16384 2023-12-11T19:37:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.681+0000 7f892d0f8b80 1 Processor -- start 2023-12-11T19:37:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 -- start start 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c977b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 0x56173c9771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c967720 0x56173c9826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c966fa0 con 0x56173c966830 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c961ee0 con 0x56173c965ed0 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c967350 con 0x56173c967720 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 0x56173c9771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 0x56173c9771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52480/0 (socket says 172.21.15.66:52480) 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 -- 172.21.15.66:0/1300664303 learned_addr learned my addr 172.21.15.66:0/1300664303 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89137fe700 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c967720 0x56173c9826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f8912ffd700 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c977b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 -- 172.21.15.66:0/1300664303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c967720 msgr2=0x56173c9826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c967720 0x56173c9826f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 -- 172.21.15.66:0/1300664303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 msgr2=0x56173c977b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c977b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.682+0000 7f89127fc700 1 -- 172.21.15.66:0/1300664303 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173c6ccc00 con 0x56173c966830 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.683+0000 7f89127fc700 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 0x56173c9771e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8908000c00 tx=0x7f890800cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e28bfc0b20d128 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.683+0000 7f88f9ffb700 1 -- 172.21.15.66:0/1300664303 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890809f510 con 0x56173c966830 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.683+0000 7f88f9ffb700 1 -- 172.21.15.66:0/1300664303 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89080b3070 con 0x56173c966830 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.683+0000 7f88f9ffb700 1 -- 172.21.15.66:0/1300664303 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890809ab10 con 0x56173c966830 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.683+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1300664303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 msgr2=0x56173c9771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.683+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 0x56173c9771e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8908000c00 tx=0x7f890800cfd0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1300664303 shutdown_connections 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c965ed0 0x56173c977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c967720 0x56173c9826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/1300664303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c966830 0x56173c9771e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1300664303 >> 172.21.15.66:0/1300664303 conn(0x56173c895780 msgr2=0x56173c96f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1300664303 shutdown_connections 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 -- 172.21.15.66:0/1300664303 wait complete. 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 -- start start 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.684+0000 7f892d0f8b80 1 Processor -- start 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c965ed0 0x56173c974c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c975190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c967720 0x56173c9760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c9765f0 con 0x56173c965ed0 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c976760 con 0x56173c967720 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173c976920 con 0x56173c966830 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f892d0f8b80 10 client.0 _ll_register_callbacks cb 0x56173c79fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f8912ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c965ed0 0x56173c974c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f89127fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c975190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f89127fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c975190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:52496/0 (socket says 172.21.15.66:52496) 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f89127fc700 1 -- 172.21.15.66:0/338054329 learned_addr learned my addr 172.21.15.66:0/338054329 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f89137fe700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c967720 0x56173c9760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f8912ffd700 1 -- 172.21.15.66:0/338054329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 msgr2=0x56173c975190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f8912ffd700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c975190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f8912ffd700 1 -- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c967720 msgr2=0x56173c9760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f8912ffd700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c967720 0x56173c9760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f8912ffd700 1 -- 172.21.15.66:0/338054329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56173c6e3d80 con 0x56173c965ed0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.685+0000 7f89127fc700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c975190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f8912ffd700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c965ed0 0x56173c974c50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x56173c97e670 tx=0x7f890400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd3b8b662bc3a3b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f88f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f88f6ff5700 1 -- 172.21.15.66:0/338054329 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904020040 con 0x56173c965ed0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f890400be90 con 0x56173c965ed0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f88f6ff5700 1 -- 172.21.15.66:0/338054329 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8904011dd0 con 0x56173c965ed0 2023-12-11T19:37:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f88f6ff5700 1 -- 172.21.15.66:0/338054329 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8904027630 con 0x56173c965ed0 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f892d0f8b80 10 client.16353 fetch_fsmap learned FSMap version 30 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.686+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56173c70a1f0 con 0x56173c965ed0 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f88f6ff5700 1 -- 172.21.15.66:0/338054329 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f890400be90 con 0x56173c965ed0 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f88f6ff5700 1 -- 172.21.15.66:0/338054329 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f89040050b0 con 0x56173c965ed0 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f892d0f8b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 30 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f892d0f8b80 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f892d0f8b80 20 client.16353 populate_metadata read hostname 'smithi066' 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c9d2a70 0x56173c9d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f892d0f8b80 4 client.16353 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56173c880f70 con 0x56173c9d2a70 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.687+0000 7f89127fc700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c9d2a70 0x56173c9d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.688+0000 7f89127fc700 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c9d2a70 0x56173c9d4e90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273b95bf54dab1c0 server_cookie=ae3cb42bedc158a2 in_seq=0 out_seq=0 2023-12-11T19:37:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.688+0000 7f88f6ff5700 10 client.16353 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f88f6ff5700 1 -- 172.21.15.66:0/338054329 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56173c880f70 con 0x56173c9d2a70 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f88f6ff5700 10 client.16353 handle_command_reply: tid=0 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f892d0f8b80 1 client.16353 shutdown 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c965ed0 msgr2=0x56173c974c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c965ed0 0x56173c974c50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x56173c97e670 tx=0x7f890400fba0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 shutdown_connections 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.697+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x56173c9d2a70 0x56173c9d4e90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56173c967720 0x56173c9760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56173c965ed0 0x56173c974c50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 1 --2- 172.21.15.66:0/338054329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56173c966830 0x56173c975190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f88f6ff5700 0 client.16353 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 >> 172.21.15.66:0/338054329 conn(0x56173c895780 msgr2=0x56173c9826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 shutdown_connections 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 1 -- 172.21.15.66:0/338054329 wait complete. 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f892d0f8b80 20 client.16353 trim_cache size 0 max 16384 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88fc0a1500 msgr2=0x7f88fc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f89197ff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88fc0a1500 0x7f88fc0a39c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56173c966f00 tx=0x7f890c009920 comp rx=0 tx=0).stop 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 msgr2=0x7f89141029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.698+0000 7f89197ff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f89141029f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8908009990 tx=0x7f890800fa00 comp rx=0 tx=0).stop 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 shutdown_connections 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f88fc0a1500 0x7f88fc0a39c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8914100420 0x7f8914104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8914101fb0 0x7f8914102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 --2- 172.21.15.66:0/38339445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8914101610 0x7f89141029f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 >> 172.21.15.66:0/38339445 conn(0x7f89140fb990 msgr2=0x7f89140688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 shutdown_connections 2023-12-11T19:37:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:04.699+0000 7f89197ff700 1 -- 172.21.15.66:0/38339445 wait complete. 2023-12-11T19:37:06.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:05 smithi167 ceph-mon[102750]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 538 MiB/s wr, 146 op/s 2023-12-11T19:37:06.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:05 smithi066 ceph-mon[97920]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 538 MiB/s wr, 146 op/s 2023-12-11T19:37:06.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:05 smithi070 ceph-mon[104044]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 538 MiB/s wr, 146 op/s 2023-12-11T19:37:08.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:07 smithi167 ceph-mon[102750]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 471 MiB/s wr, 131 op/s 2023-12-11T19:37:08.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:07 smithi066 ceph-mon[97920]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 471 MiB/s wr, 131 op/s 2023-12-11T19:37:08.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:07 smithi070 ceph-mon[104044]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 471 MiB/s wr, 131 op/s 2023-12-11T19:37:10.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:09 smithi167 ceph-mon[102750]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 388 MiB/s wr, 107 op/s 2023-12-11T19:37:10.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:09 smithi066 ceph-mon[97920]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 388 MiB/s wr, 107 op/s 2023-12-11T19:37:10.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:09 smithi070 ceph-mon[104044]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 388 MiB/s wr, 107 op/s 2023-12-11T19:37:11.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:10 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:37:11.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:10 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:37:11.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:10 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:37:11.836 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:37:11.837 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 readers = 6009000.00 kB/sec 2023-12-11T19:37:11.837 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 readers = 6005837.57 kB/sec 2023-12-11T19:37:11.837 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 6009000.00 kB/sec 2023-12-11T19:37:11.837 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 6009000.00 kB/sec 2023-12-11T19:37:11.837 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 6009000.00 kB/sec 2023-12-11T19:37:11.837 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 10485760.00 kB 2023-12-11T19:37:12.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:11 smithi167 ceph-mon[102750]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 328 MiB/s wr, 95 op/s 2023-12-11T19:37:12.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:37:12.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:37:12.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:37:12.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:11 smithi066 ceph-mon[97920]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 328 MiB/s wr, 95 op/s 2023-12-11T19:37:12.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:37:12.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:37:12.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:37:12.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:11 smithi070 ceph-mon[104044]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 328 MiB/s wr, 95 op/s 2023-12-11T19:37:12.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:37:12.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:37:12.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:37:13.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:12 smithi167 ceph-mon[102750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:37:13.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:12 smithi066 ceph-mon[97920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:37:13.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:12 smithi070 ceph-mon[104044]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:37:14.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:13 smithi167 ceph-mon[102750]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 221 MiB/s wr, 66 op/s 2023-12-11T19:37:14.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:13 smithi066 ceph-mon[97920]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 221 MiB/s wr, 66 op/s 2023-12-11T19:37:14.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:13 smithi070 ceph-mon[104044]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 221 MiB/s wr, 66 op/s 2023-12-11T19:37:16.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:15 smithi167 ceph-mon[102750]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 173 MiB/s wr, 55 op/s 2023-12-11T19:37:16.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:15 smithi066 ceph-mon[97920]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 173 MiB/s wr, 55 op/s 2023-12-11T19:37:16.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:15 smithi070 ceph-mon[104044]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 173 MiB/s wr, 55 op/s 2023-12-11T19:37:16.534 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:37:16.534 INFO:tasks.workunit.client.0.smithi066.stdout: Children see throughput for 1 re-readers = 4055304.25 kB/sec 2023-12-11T19:37:16.534 INFO:tasks.workunit.client.0.smithi066.stdout: Parent sees throughput for 1 re-readers = 4053944.91 kB/sec 2023-12-11T19:37:16.534 INFO:tasks.workunit.client.0.smithi066.stdout: Min throughput per process = 4055304.25 kB/sec 2023-12-11T19:37:16.534 INFO:tasks.workunit.client.0.smithi066.stdout: Max throughput per process = 4055304.25 kB/sec 2023-12-11T19:37:16.534 INFO:tasks.workunit.client.0.smithi066.stdout: Avg throughput per process = 4055304.25 kB/sec 2023-12-11T19:37:16.535 INFO:tasks.workunit.client.0.smithi066.stdout: Min xfer = 10485760.00 kB 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.861+0000 7fbe86dff700 1 -- 172.21.15.66:0/2764428018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 msgr2=0x7fbe800744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.861+0000 7fbe86dff700 1 --2- 172.21.15.66:0/2764428018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe800744b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009990 tx=0x7fbe6c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 -- 172.21.15.66:0/2764428018 shutdown_connections 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 --2- 172.21.15.66:0/2764428018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 0x7fbe80073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 --2- 172.21.15.66:0/2764428018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe801093f0 0x7fbe8010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 --2- 172.21.15.66:0/2764428018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe800744b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 -- 172.21.15.66:0/2764428018 >> 172.21.15.66:0/2764428018 conn(0x7fbe800fcaa0 msgr2=0x7fbe800feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 -- 172.21.15.66:0/2764428018 shutdown_connections 2023-12-11T19:37:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.862+0000 7fbe86dff700 1 -- 172.21.15.66:0/2764428018 wait complete. 2023-12-11T19:37:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 Processor -- start 2023-12-11T19:37:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 -- start start 2023-12-11T19:37:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe80199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 0x7fbe80199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe801093f0 0x7fbe801a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801a1180 con 0x7fbe80074030 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801a12c0 con 0x7fbe80075610 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe86dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801a1450 con 0x7fbe801093f0 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.863+0000 7fbe84b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe80199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe80199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60306/0 (socket says 172.21.15.66:60306) 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 -- 172.21.15.66:0/956248165 learned_addr learned my addr 172.21.15.66:0/956248165 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe8539c700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe801093f0 0x7fbe801a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 0x7fbe80199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 -- 172.21.15.66:0/956248165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe801093f0 msgr2=0x7fbe801a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe801093f0 0x7fbe801a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 -- 172.21.15.66:0/956248165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 msgr2=0x7fbe80199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 0x7fbe80199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 -- 172.21.15.66:0/956248165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0095f0 con 0x7fbe80074030 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 0x7fbe80199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe80199650 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e9a0 tx=0x7fbe7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.864+0000 7fbe7dffb700 1 -- 172.21.15.66:0/956248165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74009790 con 0x7fbe80074030 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.865+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe801a1730 con 0x7fbe80074030 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.865+0000 7fbe7dffb700 1 -- 172.21.15.66:0/956248165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7400cbd0 con 0x7fbe80074030 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.865+0000 7fbe7dffb700 1 -- 172.21.15.66:0/956248165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74018620 con 0x7fbe80074030 2023-12-11T19:37:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.865+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe801a1c40 con 0x7fbe80074030 2023-12-11T19:37:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.868+0000 7fbe7dffb700 1 -- 172.21.15.66:0/956248165 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fbe7400cd40 con 0x7fbe80074030 2023-12-11T19:37:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.869+0000 7fbe9a6b1b80 1 Processor -- start 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.869+0000 7fbe7dffb700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1480 0x7fbe700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.869+0000 7fbe7dffb700 1 -- 172.21.15.66:0/956248165 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe74014070 con 0x7fbe80074030 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.869+0000 7fbe9a6b1b80 1 -- start start 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe8539c700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1480 0x7fbe700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb7495140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb7495ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb73efaf0 con 0x555eb7495720 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb73ed410 con 0x555eb7494830 2023-12-11T19:37:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb73e8d00 con 0x555eb7493ed0 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe8539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb7495ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe8539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb7495ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60322/0 (socket says 172.21.15.66:60322) 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe8539c700 1 -- 172.21.15.66:0/1607579145 learned_addr learned my addr 172.21.15.66:0/1607579145 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb7495140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.870+0000 7fbe8539c700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1480 0x7fbe700a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe78003f60 tx=0x7fbe78003f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe8539c700 1 -- 172.21.15.66:0/1607579145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 msgr2=0x555eb74942f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe8539c700 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74942f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe8539c700 1 -- 172.21.15.66:0/1607579145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 msgr2=0x555eb7495140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe8539c700 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb7495140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe8539c700 1 -- 172.21.15.66:0/1607579145 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb72655d0 con 0x555eb7495720 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe8539c700 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb7495ba0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbe78004730 tx=0x7fbe7801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37ca08f88328cde server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe6affd700 1 -- 172.21.15.66:0/1607579145 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78007b90 con 0x555eb7495720 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe6affd700 1 -- 172.21.15.66:0/1607579145 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7800f040 con 0x555eb7495720 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.871+0000 7fbe6affd700 1 -- 172.21.15.66:0/1607579145 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78021650 con 0x555eb7495720 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1607579145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 msgr2=0x555eb7495ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb7495ba0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbe78004730 tx=0x7fbe7801b000 comp rx=0 tx=0).stop 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1607579145 shutdown_connections 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb7495ba0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1607579145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb7495140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1607579145 >> 172.21.15.66:0/1607579145 conn(0x555eb73c3780 msgr2=0x555eb7490880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1607579145 shutdown_connections 2023-12-11T19:37:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.872+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1607579145 wait complete. 2023-12-11T19:37:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 -- start start 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 Processor -- start 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7493ed0 0x555eb749df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7494830 0x555eb749e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb74a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb73a2500 con 0x555eb7495720 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb73a21a0 con 0x555eb7493ed0 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb72678c0 con 0x555eb7494830 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.873+0000 7fbe9a6b1b80 10 client.0 _ll_register_callbacks cb 0x555eb72cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe8539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb74a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe7ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7494830 0x555eb749e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7493ed0 0x555eb749df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe8539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb74a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60330/0 (socket says 172.21.15.66:60330) 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe7ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7494830 0x555eb749e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.66:60850/0 (socket says 172.21.15.66:60850) 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe8539c700 1 -- 172.21.15.66:0/120550534 learned_addr learned my addr 172.21.15.66:0/120550534 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 -- 172.21.15.66:0/120550534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7494830 msgr2=0x555eb749e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7494830 0x555eb749e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 -- 172.21.15.66:0/120550534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 msgr2=0x555eb74a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb74a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 -- 172.21.15.66:0/120550534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe78004410 con 0x555eb7493ed0 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe8539c700 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb74a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7493ed0 0x555eb749df30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe74099f30 tx=0x7fbe7409a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3476ee2b0641928b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.874+0000 7fbe5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe5ffff700 1 -- 172.21.15.66:0/120550534 <== mon.2 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7409a230 con 0x555eb7493ed0 2023-12-11T19:37:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eb7487730 con 0x555eb7493ed0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe5ffff700 1 -- 172.21.15.66:0/120550534 <== mon.2 v2:172.21.15.70:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe7409e380 con 0x555eb7493ed0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe5ffff700 1 -- 172.21.15.66:0/120550534 <== mon.2 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe740a25c0 con 0x555eb7493ed0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe9a6b1b80 10 client.26057 fetch_fsmap learned FSMap version 31 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555eb7211d80 con 0x555eb7493ed0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe5ffff700 1 -- 172.21.15.66:0/120550534 <== mon.2 v2:172.21.15.70:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555eb7487730 con 0x555eb7493ed0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe5ffff700 1 -- 172.21.15.66:0/120550534 <== mon.2 v2:172.21.15.70:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbe74096020 con 0x555eb7493ed0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe9a6b1b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 31 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.875+0000 7fbe9a6b1b80 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.876+0000 7fbe9a6b1b80 20 client.26057 populate_metadata read hostname 'smithi066' 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.876+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb74a9d70 0x555eb74ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.876+0000 7fbe9a6b1b80 4 client.26057 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.876+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eb7463ee0 con 0x555eb74a9d70 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.876+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb74a9d70 0x555eb74ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.877+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb74a9d70 0x555eb74ac190 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ff85f8956e9d49 server_cookie=39a5bd2179b18e9d in_seq=0 out_seq=0 2023-12-11T19:37:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.877+0000 7fbe5ffff700 10 client.26057 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.877+0000 7fbe5ffff700 1 -- 172.21.15.66:0/120550534 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555eb7463ee0 con 0x555eb74a9d70 2023-12-11T19:37:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.877+0000 7fbe5ffff700 10 client.26057 handle_command_reply: tid=0 2023-12-11T19:37:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.877+0000 7fbe9a6b1b80 1 client.26057 shutdown 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7493ed0 msgr2=0x555eb749df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7493ed0 0x555eb749df30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe74099f30 tx=0x7fbe7409a770 comp rx=0 tx=0).stop 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 shutdown_connections 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb74a9d70 0x555eb74ac190 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7493ed0 0x555eb749df30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe5ffff700 0 client.26057 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7495720 0x555eb74a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/120550534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7494830 0x555eb749e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 >> 172.21.15.66:0/120550534 conn(0x555eb73c3780 msgr2=0x555eb748eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 shutdown_connections 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/120550534 wait complete. 2023-12-11T19:37:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.878+0000 7fbe9a6b1b80 20 client.26057 trim_cache size 0 max 16384 2023-12-11T19:37:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 Processor -- start 2023-12-11T19:37:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 -- start start 2023-12-11T19:37:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 0x555eb749d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb74a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7495720 0x555eb7495b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb748fd70 con 0x555eb7493ed0 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb7494670 con 0x555eb7494830 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.894+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb7495350 con 0x555eb7495720 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 0x555eb749d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 0x555eb749d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60346/0 (socket says 172.21.15.66:60346) 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 -- 172.21.15.66:0/704907194 learned_addr learned my addr 172.21.15.66:0/704907194 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe8539c700 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7495720 0x555eb7495b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb74a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 -- 172.21.15.66:0/704907194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7495720 msgr2=0x555eb7495b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7495720 0x555eb7495b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 -- 172.21.15.66:0/704907194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 msgr2=0x555eb74a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb74a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 -- 172.21.15.66:0/704907194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eb71fac00 con 0x555eb7493ed0 2023-12-11T19:37:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 0x555eb749d2a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbe74096ca0 tx=0x7fbe74099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f559a484cb1d6b40 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe6b7fe700 1 -- 172.21.15.66:0/704907194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740a25c0 con 0x555eb7493ed0 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe6b7fe700 1 -- 172.21.15.66:0/704907194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe74096020 con 0x555eb7493ed0 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.895+0000 7fbe6b7fe700 1 -- 172.21.15.66:0/704907194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74010710 con 0x555eb7493ed0 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/704907194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 msgr2=0x555eb749d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 0x555eb749d2a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbe74096ca0 tx=0x7fbe74099a80 comp rx=0 tx=0).stop 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/704907194 shutdown_connections 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7495720 0x555eb7495b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7493ed0 0x555eb749d2a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/704907194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7494830 0x555eb74a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/704907194 >> 172.21.15.66:0/704907194 conn(0x555eb73c3780 msgr2=0x555eb749bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/704907194 shutdown_connections 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.896+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/704907194 wait complete. 2023-12-11T19:37:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.897+0000 7fbe9a6b1b80 1 -- start start 2023-12-11T19:37:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.897+0000 7fbe9a6b1b80 1 Processor -- start 2023-12-11T19:37:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 0x555eb74a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7495720 0x555eb74a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb74a45f0 con 0x555eb7494830 2023-12-11T19:37:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb74a4760 con 0x555eb7495720 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eb74a4920 con 0x555eb7493ed0 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe9a6b1b80 10 client.0 _ll_register_callbacks cb 0x555eb72cda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 0x555eb74a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 0x555eb74a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60362/0 (socket says 172.21.15.66:60362) 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 -- 172.21.15.66:0/1325207029 learned_addr learned my addr 172.21.15.66:0/1325207029 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe8539c700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7495720 0x555eb74a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 -- 172.21.15.66:0/1325207029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 msgr2=0x555eb74a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 -- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7495720 msgr2=0x555eb74a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7495720 0x555eb74a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.898+0000 7fbe7ffff700 1 -- 172.21.15.66:0/1325207029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eb7211d80 con 0x555eb7494830 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe7ffff700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 0x555eb74a3190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009990 tx=0x7fbe6c005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52afa01fc5b9188 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe5ffff700 1 -- 172.21.15.66:0/1325207029 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c02b070 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe7800f040 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe5ffff700 1 -- 172.21.15.66:0/1325207029 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe6c005670 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe5ffff700 1 -- 172.21.15.66:0/1325207029 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe6c003680 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe9a6b1b80 10 client.16371 fetch_fsmap learned FSMap version 31 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555eb72381f0 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe5ffff700 1 -- 172.21.15.66:0/1325207029 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7800f040 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.899+0000 7fbe5ffff700 1 -- 172.21.15.66:0/1325207029 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbe6c00bcf0 con 0x555eb7494830 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe9a6b1b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 31 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe9a6b1b80 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe9a6b1b80 20 client.16371 populate_metadata read hostname 'smithi066' 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb7500cd0 0x555eb75030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe9a6b1b80 4 client.16371 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 --> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555eb73aef70 con 0x555eb7500cd0 2023-12-11T19:37:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.900+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb7500cd0 0x555eb75030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.901+0000 7fbe84b9b700 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb7500cd0 0x555eb75030f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbee5443c805f59e server_cookie=87c906e11ea376dd in_seq=0 out_seq=0 2023-12-11T19:37:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.901+0000 7fbe5ffff700 10 client.16371 ms_handle_connect on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.910+0000 7fbe5ffff700 1 -- 172.21.15.66:0/1325207029 <== mds.0 v2:172.21.15.70:6838/725803001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555eb73aef70 con 0x555eb7500cd0 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.910+0000 7fbe5ffff700 10 client.16371 handle_command_reply: tid=0 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.910+0000 7fbe9a6b1b80 1 client.16371 shutdown 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 msgr2=0x555eb74a3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 0x555eb74a3190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009990 tx=0x7fbe6c005200 comp rx=0 tx=0).stop 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 shutdown_connections 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:6838/725803001,v1:172.21.15.70:6839/725803001] conn(0x555eb7500cd0 0x555eb75030f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eb7495720 0x555eb74a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555eb7493ed0 0x555eb74a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 --2- 172.21.15.66:0/1325207029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555eb7494830 0x555eb74a3190 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe5ffff700 0 client.16371 ms_handle_reset on v2:172.21.15.70:6838/725803001 2023-12-11T19:37:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 >> 172.21.15.66:0/1325207029 conn(0x555eb73c3780 msgr2=0x555eb749bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 shutdown_connections 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 1 -- 172.21.15.66:0/1325207029 wait complete. 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.911+0000 7fbe9a6b1b80 20 client.16371 trim_cache size 0 max 16384 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:{ 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "message": "", 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout: "return_code": 0 2023-12-11T19:37:17.913 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stdout:} 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1480 msgr2=0x7fbe700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1480 0x7fbe700a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe78003f60 tx=0x7fbe78003f90 comp rx=0 tx=0).stop 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 msgr2=0x7fbe80199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe80199650 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e9a0 tx=0x7fbe7400ecb0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 shutdown_connections 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fbe700a1480 0x7fbe700a3940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe801093f0 0x7fbe801a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe80074030 0x7fbe80199650 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 --2- 172.21.15.66:0/956248165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe80075610 0x7fbe80199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:17.914 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 >> 172.21.15.66:0/956248165 conn(0x7fbe800fcaa0 msgr2=0x7fbe801052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 shutdown_connections 2023-12-11T19:37:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi066.front.sepia.ceph.com.smithi066.stderr:2023-12-11T19:37:17.912+0000 7fbe86dff700 1 -- 172.21.15.66:0/956248165 wait complete. 2023-12-11T19:37:18.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:17 smithi167 ceph-mon[102750]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 58 MiB/s wr, 28 op/s 2023-12-11T19:37:18.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:17 smithi066 ceph-mon[97920]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 58 MiB/s wr, 28 op/s 2023-12-11T19:37:18.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:17 smithi070 ceph-mon[104044]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 58 MiB/s wr, 28 op/s 2023-12-11T19:37:20.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:19 smithi167 ceph-mon[102750]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.0 MiB/s wr, 11 op/s 2023-12-11T19:37:20.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:19 smithi066 ceph-mon[97920]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.0 MiB/s wr, 11 op/s 2023-12-11T19:37:20.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:19 smithi070 ceph-mon[104044]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.0 MiB/s wr, 11 op/s 2023-12-11T19:37:22.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:21 smithi167 ceph-mon[102750]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2023-12-11T19:37:22.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:21 smithi066 ceph-mon[97920]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2023-12-11T19:37:22.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:21 smithi070 ceph-mon[104044]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2023-12-11T19:37:24.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:23 smithi167 ceph-mon[102750]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-11T19:37:24.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:23 smithi066 ceph-mon[97920]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-11T19:37:24.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:23 smithi070 ceph-mon[104044]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-11T19:37:26.205 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:37:26.205 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:37:26.205 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-12-11T19:37:26.205 INFO:tasks.workunit.client.0.smithi066.stdout:iozone test complete. 2023-12-11T19:37:26.209 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-11T19:37:26.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-11T19:37:26.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:25 smithi167 ceph-mon[102750]: pgmap v475: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2023-12-11T19:37:26.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:25 smithi066 ceph-mon[97920]: pgmap v475: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2023-12-11T19:37:26.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:25 smithi070 ceph-mon[104044]: pgmap v475: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2023-12-11T19:37:26.367 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2023-12-11T19:37:26.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-11T19:37:27.327 DEBUG:teuthology.parallel:result is None 2023-12-11T19:37:27.328 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-12-11T19:37:27.414 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-12-11T19:37:27.415 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-12-11T19:37:27.426 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:37:27.438 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-12-11T19:37:27.496 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-12-11T19:37:27.496 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-12-11T19:37:27.497 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-11T19:37:27.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:37:27.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 -- 172.21.15.66:0/1815728966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8103030 msgr2=0x7f2ed81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:27.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/1815728966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8103030 0x7f2ed81098e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b0d0 tx=0x7f2ed000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 -- 172.21.15.66:0/1815728966 shutdown_connections 2023-12-11T19:37:27.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/1815728966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8103030 0x7f2ed81098e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/1815728966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ed81014a0 0x7f2ed81018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/1815728966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8102690 0x7f2ed8102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 -- 172.21.15.66:0/1815728966 >> 172.21.15.66:0/1815728966 conn(0x7f2ed80fca10 msgr2=0x7f2ed80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:27.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 -- 172.21.15.66:0/1815728966 shutdown_connections 2023-12-11T19:37:27.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.714+0000 7f2ede1ff700 1 -- 172.21.15.66:0/1815728966 wait complete. 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.715+0000 7f2ede1ff700 1 Processor -- start 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.715+0000 7f2ede1ff700 1 -- start start 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ede1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ed81014a0 0x7f2ed81995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ede1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 0x7f2ed8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ede1ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8103030 0x7f2ed81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ede1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a1120 con 0x7f2ed8102690 2023-12-11T19:37:27.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ede1ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a1260 con 0x7f2ed8103030 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ede1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a13f0 con 0x7f2ed81014a0 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 0x7f2ed8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 0x7f2ed8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50876/0 (socket says 172.21.15.66:50876) 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 -- 172.21.15.66:0/156182266 learned_addr learned my addr 172.21.15.66:0/156182266 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed77fe700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ed81014a0 0x7f2ed81995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed7fff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8103030 0x7f2ed81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 -- 172.21.15.66:0/156182266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ed81014a0 msgr2=0x7f2ed81995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ed81014a0 0x7f2ed81995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 -- 172.21.15.66:0/156182266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8103030 msgr2=0x7f2ed81a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:27.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8103030 0x7f2ed81a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.716+0000 7f2ed6ffd700 1 -- 172.21.15.66:0/156182266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009d00 con 0x7f2ed8102690 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.717+0000 7f2ed6ffd700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 0x7f2ed8199b30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00e9a0 tx=0x7f2ecc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.717+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc009790 con 0x7f2ed8102690 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.717+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ecc010430 con 0x7f2ed8102690 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.717+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed81a16d0 con 0x7f2ed8102690 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.717+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00cea0 con 0x7f2ed8102690 2023-12-11T19:37:27.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.717+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed81a1c30 con 0x7f2ed8102690 2023-12-11T19:37:27.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.718+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed8068230 con 0x7f2ed8102690 2023-12-11T19:37:27.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.720+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2ecc0105a0 con 0x7f2ed8102690 2023-12-11T19:37:27.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.722+0000 7f2ed4ff9700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2ec00a1500 0x7f2ec00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:27.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.722+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2ecc014070 con 0x7f2ed8102690 2023-12-11T19:37:27.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.722+0000 7f2ed77fe700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2ec00a1500 0x7f2ec00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:27.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.723+0000 7f2ed77fe700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2ec00a1500 0x7f2ec00a39c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2ed81024f0 tx=0x7f2ec8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:27.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.726+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2ecc096480 con 0x7f2ed8102690 2023-12-11T19:37:27.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.941+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ed81a2020 con 0x7f2ed8102690 2023-12-11T19:37:27.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.943+0000 7f2ed4ff9700 1 -- 172.21.15.66:0/156182266 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f2ecc099aa0 con 0x7f2ed8102690 2023-12-11T19:37:27.944 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:37:27.944 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:37:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.945+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2ec00a1500 msgr2=0x7f2ec00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:27.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.945+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2ec00a1500 0x7f2ec00a39c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2ed81024f0 tx=0x7f2ec8009260 comp rx=0 tx=0).stop 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.945+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 msgr2=0x7f2ed8199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.945+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 0x7f2ed8199b30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00e9a0 tx=0x7f2ecc00ecb0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 shutdown_connections 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2ec00a1500 0x7f2ec00a39c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8103030 0x7f2ed81a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ed81014a0 0x7f2ed81995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 --2- 172.21.15.66:0/156182266 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2ed8102690 0x7f2ed8199b30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 >> 172.21.15.66:0/156182266 conn(0x7f2ed80fca10 msgr2=0x7f2ed8073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 shutdown_connections 2023-12-11T19:37:27.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:27.946+0000 7f2ede1ff700 1 -- 172.21.15.66:0/156182266 wait complete. 2023-12-11T19:37:27.966 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:37:27.966 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:37:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.175+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3407190222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 msgr2=0x7f7fdc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3407190222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc1043f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009990 tx=0x7f7fd8009ca0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3407190222 shutdown_connections 2023-12-11T19:37:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3407190222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc107260 0x7f7fdc1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3407190222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc1043f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3407190222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc104930 0x7f7fdc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3407190222 >> 172.21.15.66:0/3407190222 conn(0x7f7fdc0fb9f0 msgr2=0x7f7fdc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3407190222 shutdown_connections 2023-12-11T19:37:28.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.176+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3407190222 wait complete. 2023-12-11T19:37:28.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 Processor -- start 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 -- start start 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc104930 0x7f7fdc197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc107260 0x7f7fdc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc19ee50 con 0x7f7fdc102000 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc19ef90 con 0x7f7fdc107260 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.177+0000 7f7fe41ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc19f120 con 0x7f7fdc104930 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50892/0 (socket says 172.21.15.66:50892) 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 -- 172.21.15.66:0/3021107760 learned_addr learned my addr 172.21.15.66:0/3021107760 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:28.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe279c700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc107260 0x7f7fdc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe179a700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc104930 0x7f7fdc197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 -- 172.21.15.66:0/3021107760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc104930 msgr2=0x7f7fdc197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc104930 0x7f7fdc197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 -- 172.21.15.66:0/3021107760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc107260 msgr2=0x7f7fdc19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc107260 0x7f7fdc19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 -- 172.21.15.66:0/3021107760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd80095f0 con 0x7f7fdc102000 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe179a700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc104930 0x7f7fdc197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.178+0000 7f7fe1f9b700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc197320 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8000c00 tx=0x7f7fd80047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.179+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd801d070 con 0x7f7fdc102000 2023-12-11T19:37:28.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.179+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd80041b0 con 0x7f7fdc102000 2023-12-11T19:37:28.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.179+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc19f3a0 con 0x7f7fdc102000 2023-12-11T19:37:28.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.179+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8022e70 con 0x7f7fdc102000 2023-12-11T19:37:28.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.179+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc19f820 con 0x7f7fdc102000 2023-12-11T19:37:28.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.180+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc0fd570 con 0x7f7fdc102000 2023-12-11T19:37:28.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.182+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7fd8022600 con 0x7f7fdc102000 2023-12-11T19:37:28.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.183+0000 7f7fd2ffd700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7fc80a1500 0x7f7fc80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.184+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fd80ce270 con 0x7f7fdc102000 2023-12-11T19:37:28.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.184+0000 7f7fe179a700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7fc80a1500 0x7f7fc80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.184+0000 7f7fe179a700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7fc80a1500 0x7f7fc80a39c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005fd0 tx=0x7f7fcc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:28.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.185+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7fd8097920 con 0x7f7fdc102000 2023-12-11T19:37:28.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:27 smithi167 ceph-mon[102750]: pgmap v476: 97 pgs: 97 active+clean; 5.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 107 op/s 2023-12-11T19:37:28.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:27 smithi066 ceph-mon[97920]: pgmap v476: 97 pgs: 97 active+clean; 5.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 107 op/s 2023-12-11T19:37:28.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:27 smithi070 ceph-mon[104044]: pgmap v476: 97 pgs: 97 active+clean; 5.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 107 op/s 2023-12-11T19:37:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.401+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fdc10cc80 con 0x7f7fdc102000 2023-12-11T19:37:28.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.402+0000 7f7fd2ffd700 1 -- 172.21.15.66:0/3021107760 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fd809af40 con 0x7f7fdc102000 2023-12-11T19:37:28.403 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:37:28.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.404+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7fc80a1500 msgr2=0x7f7fc80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.404+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7fc80a1500 0x7f7fc80a39c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005fd0 tx=0x7f7fcc005e30 comp rx=0 tx=0).stop 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.404+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 msgr2=0x7f7fdc197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.404+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc197320 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8000c00 tx=0x7f7fd80047a0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 shutdown_connections 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f7fc80a1500 0x7f7fc80a39c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fdc107260 0x7f7fdc19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fdc102000 0x7f7fdc197320 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 --2- 172.21.15.66:0/3021107760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fdc104930 0x7f7fdc197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 >> 172.21.15.66:0/3021107760 conn(0x7f7fdc0fb9f0 msgr2=0x7f7fdc102940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 shutdown_connections 2023-12-11T19:37:28.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.405+0000 7f7fe41ff700 1 -- 172.21.15.66:0/3021107760 wait complete. 2023-12-11T19:37:28.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:37:28.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 -- 172.21.15.66:0/247484856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00601015a0 msgr2=0x7f00601019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 --2- 172.21.15.66:0/247484856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00601015a0 0x7f00601019c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f005c0099c0 tx=0x7f005c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 -- 172.21.15.66:0/247484856 shutdown_connections 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 --2- 172.21.15.66:0/247484856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0060103130 0x7f00601099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 --2- 172.21.15.66:0/247484856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00601015a0 0x7f00601019c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 --2- 172.21.15.66:0/247484856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060102790 0x7f0060102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 -- 172.21.15.66:0/247484856 >> 172.21.15.66:0/247484856 conn(0x7f00600fcb10 msgr2=0x7f00600fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.635+0000 7f00681ff700 1 -- 172.21.15.66:0/247484856 shutdown_connections 2023-12-11T19:37:28.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.636+0000 7f00681ff700 1 -- 172.21.15.66:0/247484856 wait complete. 2023-12-11T19:37:28.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.636+0000 7f00681ff700 1 Processor -- start 2023-12-11T19:37:28.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 -- start start 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00601015a0 0x7f0060199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 0x7f0060199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060103130 0x7f00601a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a1150 con 0x7f0060102790 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a1290 con 0x7f00601015a0 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f00681ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a1420 con 0x7f0060103130 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f006579a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 0x7f0060199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f006579a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 0x7f0060199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50896/0 (socket says 172.21.15.66:50896) 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.637+0000 7f006579a700 1 -- 172.21.15.66:0/1718455466 learned_addr learned my addr 172.21.15.66:0/1718455466 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f0065f9b700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00601015a0 0x7f0060199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006679c700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060103130 0x7f00601a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006579a700 1 -- 172.21.15.66:0/1718455466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060103130 msgr2=0x7f00601a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006579a700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060103130 0x7f00601a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006579a700 1 -- 172.21.15.66:0/1718455466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00601015a0 msgr2=0x7f0060199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006579a700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00601015a0 0x7f0060199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006579a700 1 -- 172.21.15.66:0/1718455466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0095f0 con 0x7f0060102790 2023-12-11T19:37:28.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f006579a700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 0x7f0060199bb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f005000c9b0 tx=0x7f005000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:28.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050007b00 con 0x7f0060102790 2023-12-11T19:37:28.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f005000ce50 con 0x7f0060102790 2023-12-11T19:37:28.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00601a1700 con 0x7f0060102790 2023-12-11T19:37:28.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.638+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050018420 con 0x7f0060102790 2023-12-11T19:37:28.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.639+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00601a1c10 con 0x7f0060102790 2023-12-11T19:37:28.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.640+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0060068310 con 0x7f0060102790 2023-12-11T19:37:28.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.642+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f005000f450 con 0x7f0060102790 2023-12-11T19:37:28.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.643+0000 7f0056ffd700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f004c0a1480 0x7f004c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:28.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.643+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f00500ccca0 con 0x7f0060102790 2023-12-11T19:37:28.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.643+0000 7f0065f9b700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f004c0a1480 0x7f004c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:28.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.644+0000 7f0065f9b700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f004c0a1480 0x7f004c0a3940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f005c006010 tx=0x7f005c003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:28.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.647+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f00500962c0 con 0x7f0060102790 2023-12-11T19:37:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.862+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00601a1f30 con 0x7f0060102790 2023-12-11T19:37:28.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.862+0000 7f0056ffd700 1 -- 172.21.15.66:0/1718455466 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f005000f700 con 0x7f0060102790 2023-12-11T19:37:28.863 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:37:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f004c0a1480 msgr2=0x7f004c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f004c0a1480 0x7f004c0a3940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f005c006010 tx=0x7f005c003740 comp rx=0 tx=0).stop 2023-12-11T19:37:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 msgr2=0x7f0060199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 0x7f0060199bb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f005000c9b0 tx=0x7f005000ccc0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 shutdown_connections 2023-12-11T19:37:28.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f004c0a1480 0x7f004c0a3940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00601015a0 0x7f0060199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0060103130 0x7f00601a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 --2- 172.21.15.66:0/1718455466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0060102790 0x7f0060199bb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 >> 172.21.15.66:0/1718455466 conn(0x7f00600fcb10 msgr2=0x7f0060073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 shutdown_connections 2023-12-11T19:37:28.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:28.865+0000 7f00681ff700 1 -- 172.21.15.66:0/1718455466 wait complete. 2023-12-11T19:37:28.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 -- 172.21.15.66:0/17912409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014c0 msgr2=0x7f64f8101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 --2- 172.21.15.66:0/17912409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014c0 0x7f64f8101920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0099c0 tx=0x7f64ec009cd0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 -- 172.21.15.66:0/17912409 shutdown_connections 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 --2- 172.21.15.66:0/17912409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8101e60 0x7f64f8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 --2- 172.21.15.66:0/17912409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81002d0 0x7f64f81006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 --2- 172.21.15.66:0/17912409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81014c0 0x7f64f8101920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 -- 172.21.15.66:0/17912409 >> 172.21.15.66:0/17912409 conn(0x7f64f80fb820 msgr2=0x7f64f80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 -- 172.21.15.66:0/17912409 shutdown_connections 2023-12-11T19:37:29.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.098+0000 7f64f759e700 1 -- 172.21.15.66:0/17912409 wait complete. 2023-12-11T19:37:29.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.099+0000 7f64f759e700 1 Processor -- start 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.099+0000 7f64f759e700 1 -- start start 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.099+0000 7f64f759e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 0x7f64f8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.099+0000 7f64f759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81014c0 0x7f64f8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.099+0000 7f64f759e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8101e60 0x7f64f81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f759e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f81a1070 con 0x7f64f81002d0 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f759e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f81a11b0 con 0x7f64f8101e60 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f759e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f81a1340 con 0x7f64f81014c0 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 0x7f64f8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 0x7f64f8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50918/0 (socket says 172.21.15.66:50918) 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 -- 172.21.15.66:0/1612470368 learned_addr learned my addr 172.21.15.66:0/1612470368 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f5d9b700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81014c0 0x7f64f8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 -- 172.21.15.66:0/1612470368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81014c0 msgr2=0x7f64f8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81014c0 0x7f64f8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 -- 172.21.15.66:0/1612470368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8101e60 msgr2=0x7f64f81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8101e60 0x7f64f81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 -- 172.21.15.66:0/1612470368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec0095f0 con 0x7f64f81002d0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f5d9b700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81014c0 0x7f64f8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.100+0000 7f64f659c700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 0x7f64f8199540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f64e800d980 tx=0x7f64e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.101+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800de60 con 0x7f64f81002d0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.101+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f81a1620 con 0x7f64f81002d0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.101+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64e80044b0 con 0x7f64f81002d0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.101+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8003b40 con 0x7f64f81002d0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.101+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f81a1b80 con 0x7f64f81002d0 2023-12-11T19:37:29.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.102+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f80681c0 con 0x7f64f81002d0 2023-12-11T19:37:29.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.104+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f64e8003ca0 con 0x7f64f81002d0 2023-12-11T19:37:29.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.105+0000 7f64e77fe700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e00a1500 0x7f64e00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.105+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f64e80cc9f0 con 0x7f64f81002d0 2023-12-11T19:37:29.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.105+0000 7f64f5d9b700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e00a1500 0x7f64e00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.106+0000 7f64f5d9b700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e00a1500 0x7f64e00a39c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006010 tx=0x7f64ec005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:29.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.109+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f64e8096010 con 0x7f64f81002d0 2023-12-11T19:37:29.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:28 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/156182266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:37:29.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:28 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3021107760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:37:29.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:28 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1718455466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:37:29.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:28 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/156182266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:37:29.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:28 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3021107760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:37:29.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:28 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1718455466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:37:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.323+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64f81a1ed0 con 0x7f64f81002d0 2023-12-11T19:37:29.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.324+0000 7f64e77fe700 1 -- 172.21.15.66:0/1612470368 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64e8009940 con 0x7f64f81002d0 2023-12-11T19:37:29.324 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e00a1500 msgr2=0x7f64e00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e00a1500 0x7f64e00a39c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006010 tx=0x7f64ec005c00 comp rx=0 tx=0).stop 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 msgr2=0x7f64f8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 0x7f64f8199540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f64e800d980 tx=0x7f64e800dd40 comp rx=0 tx=0).stop 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 shutdown_connections 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f64e00a1500 0x7f64e00a39c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8101e60 0x7f64f81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f81002d0 0x7f64f8199540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 --2- 172.21.15.66:0/1612470368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f81014c0 0x7f64f8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 >> 172.21.15.66:0/1612470368 conn(0x7f64f80fb820 msgr2=0x7f64f8068660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:29.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 shutdown_connections 2023-12-11T19:37:29.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.326+0000 7f64f759e700 1 -- 172.21.15.66:0/1612470368 wait complete. 2023-12-11T19:37:29.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-11T19:37:29.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:28 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/156182266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:37:29.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:28 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3021107760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:37:29.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:28 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1718455466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:37:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 -- 172.21.15.66:0/752493485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168103030 msgr2=0x7f01681098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 --2- 172.21.15.66:0/752493485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168103030 0x7f01681098e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f016000b0d0 tx=0x7f016000b3e0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 -- 172.21.15.66:0/752493485 shutdown_connections 2023-12-11T19:37:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 --2- 172.21.15.66:0/752493485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01681014a0 0x7f01681018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 --2- 172.21.15.66:0/752493485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168103030 0x7f01681098e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 --2- 172.21.15.66:0/752493485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168102690 0x7f0168102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 -- 172.21.15.66:0/752493485 >> 172.21.15.66:0/752493485 conn(0x7f01680fca10 msgr2=0x7f01680fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:29.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 -- 172.21.15.66:0/752493485 shutdown_connections 2023-12-11T19:37:29.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.552+0000 7f016e5ff700 1 -- 172.21.15.66:0/752493485 wait complete. 2023-12-11T19:37:29.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 Processor -- start 2023-12-11T19:37:29.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 -- start start 2023-12-11T19:37:29.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01681014a0 0x7f01681995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 0x7f0168199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168103030 0x7f01681a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01681a1090 con 0x7f0168102690 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01681a11d0 con 0x7f01681014a0 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.554+0000 7f016e5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01681a1360 con 0x7f0168103030 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 0x7f0168199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 0x7f0168199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50936/0 (socket says 172.21.15.66:50936) 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 -- 172.21.15.66:0/3494031452 learned_addr learned my addr 172.21.15.66:0/3494031452 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f0167fff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01681014a0 0x7f01681995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f016cb9c700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168103030 0x7f01681a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 -- 172.21.15.66:0/3494031452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168103030 msgr2=0x7f01681a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168103030 0x7f01681a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 -- 172.21.15.66:0/3494031452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01681014a0 msgr2=0x7f01681995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:29.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01681014a0 0x7f01681995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:29.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 -- 172.21.15.66:0/3494031452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160009d00 con 0x7f0168102690 2023-12-11T19:37:29.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f016cb9c700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168103030 0x7f01681a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:37:29.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.555+0000 7f01677fe700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 0x7f0168199af0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c9b0 tx=0x7f015c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:29.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.556+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c007b00 con 0x7f0168102690 2023-12-11T19:37:29.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.556+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015c007c60 con 0x7f0168102690 2023-12-11T19:37:29.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.556+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c018660 con 0x7f0168102690 2023-12-11T19:37:29.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.556+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01681a1640 con 0x7f0168102690 2023-12-11T19:37:29.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.556+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01681a1b50 con 0x7f0168102690 2023-12-11T19:37:29.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.557+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0168068230 con 0x7f0168102690 2023-12-11T19:37:29.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.559+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f015c00f450 con 0x7f0168102690 2023-12-11T19:37:29.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.560+0000 7f01657fa700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01500a1500 0x7f01500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:37:29.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.560+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f015c0ccf40 con 0x7f0168102690 2023-12-11T19:37:29.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.561+0000 7f0167fff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01500a1500 0x7f01500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:37:29.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.561+0000 7f0167fff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01500a1500 0x7f01500a39c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01681024f0 tx=0x7f0158009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:37:29.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.562+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f015c096670 con 0x7f0168102690 2023-12-11T19:37:29.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.833+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f016810cd70 con 0x7f0168102690 2023-12-11T19:37:29.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.845+0000 7f01657fa700 1 -- 172.21.15.66:0/3494031452 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733876 (secure 0 0 0) 0x7f015c00fe60 con 0x7f0168102690 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 18, 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: "active_gid": 14150, 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: "active_name": "x", 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: "active_addrs": { 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:6800", 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 775613503 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:6801", 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 775613503 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "active_addr": "172.21.15.66:6801/775613503", 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "active_change": "2023-12-11T19:24:10.191074+0000", 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "active_mgr_features": 4540138322906710015, 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2023-12-11T19:37:29.859 INFO:teuthology.orchestra.run.smithi066.stdout: "standbys": [ 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 14217, 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "y", 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr_features": 4540138322906710015, 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:29.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.862 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.863 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.864 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.865 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:29.865 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.865 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.865 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.866 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.867 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2023-12-11T19:37:29.869 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2023-12-11T19:37:29.870 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2023-12-11T19:37:29.871 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.872 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.873 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.875 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.876 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.877 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.878 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2023-12-11T19:37:29.879 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.880 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:29.881 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2023-12-11T19:37:29.882 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T19:37:29.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.884 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.885 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.886 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.886 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.886 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.886 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.886 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.887 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.888 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.889 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.889 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.889 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.889 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.889 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:29.890 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2023-12-11T19:37:29.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.892 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap" 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.893 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.894 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2023-12-11T19:37:29.895 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2023-12-11T19:37:29.896 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2023-12-11T19:37:29.897 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.899 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.900 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.901 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.902 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm_log_destination": { 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm_log_destination", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "file", 2023-12-11T19:37:29.904 INFO:teuthology.orchestra.run.smithi066.stdout: "file,syslog", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "syslog" 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.905 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.906 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.907 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.908 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2023-12-11T19:37:29.909 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.910 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.911 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.912 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.915 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2023-12-11T19:37:29.916 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.917 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_nvmeof": { 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_nvmeof", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.918 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Nvme-of container image", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.920 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.921 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.922 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.924 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.925 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2023-12-11T19:37:29.926 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.927 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.928 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.929 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.930 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "inventory_list_all": { 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "inventory_list_all", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.931 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.933 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:29.934 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:29.935 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.936 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.937 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.940 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-11T19:37:29.941 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2023-12-11T19:37:29.942 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.943 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.944 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.946 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.947 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.948 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.952 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2023-12-11T19:37:29.953 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2023-12-11T19:37:29.954 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2023-12-11T19:37:29.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2023-12-11T19:37:29.956 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.957 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.958 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:29.960 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2023-12-11T19:37:29.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2023-12-11T19:37:29.962 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:29.963 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.964 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.965 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.967 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2023-12-11T19:37:29.968 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.971 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.972 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.974 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.976 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:29.977 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.978 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.980 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.981 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.982 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.985 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.986 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.989 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.990 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-11T19:37:29.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-11T19:37:29.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:29.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.994 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.995 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.996 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.997 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.998 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:29.999 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.000 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "UNSAFE_TLS_v1_2": { 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.002 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.004 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2023-12-11T19:37:30.005 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2023-12-11T19:37:30.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.007 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.011 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.012 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.013 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.014 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.015 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.016 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.018 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.020 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.021 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.024 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.026 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.027 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.028 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.029 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.030 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.031 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2023-12-11T19:37:30.032 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2023-12-11T19:37:30.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.035 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.037 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.038 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.039 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.040 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.041 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.042 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.043 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.044 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.045 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2023-12-11T19:37:30.046 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.048 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.049 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.050 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.052 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.053 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.056 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2023-12-11T19:37:30.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2023-12-11T19:37:30.058 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.059 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.060 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.061 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.063 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.065 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.066 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.067 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.068 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.070 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.072 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.073 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.075 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.076 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.078 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.079 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.081 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.082 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.083 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.084 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.085 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2023-12-11T19:37:30.086 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-11T19:37:30.087 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.088 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.089 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.091 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.092 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.093 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.094 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.095 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.096 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.097 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.098 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.099 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.100 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.101 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.102 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.103 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.104 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.105 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.106 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.107 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.108 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.109 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.110 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.111 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.112 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.113 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.114 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.115 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.116 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.118 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.119 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.120 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.121 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.122 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.123 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.124 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.125 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.126 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.127 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2023-12-11T19:37:30.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.129 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.130 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.131 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.133 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.133 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.133 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.133 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.133 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.134 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.137 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.140 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.141 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.142 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.143 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.145 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.146 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.147 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.151 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-12-11T19:37:30.152 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.153 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.155 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.156 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.170 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.171 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.172 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.173 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.174 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.175 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.176 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "drive_group_interval": { 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "drive_group_interval", 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.177 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300.0", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.178 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.179 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.181 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.182 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.183 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.184 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.185 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.186 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.187 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.188 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.189 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.190 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.191 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.192 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.193 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.194 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.196 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2023-12-11T19:37:30.197 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2023-12-11T19:37:30.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.199 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-12-11T19:37:30.200 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.201 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.202 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.203 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.204 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.205 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.206 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2023-12-11T19:37:30.207 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.208 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.209 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.210 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.213 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.214 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.215 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.216 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.217 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-11T19:37:30.218 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.219 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.220 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.221 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.222 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.223 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.224 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.225 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.226 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.227 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.228 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.229 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-11T19:37:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "periodic_async_work": { 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "periodic_async_work", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Periodically check for async work", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2023-12-11T19:37:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2023-12-11T19:37:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2023-12-11T19:37:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 24110, 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "z", 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr_features": 4540138322906710015, 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2023-12-11T19:37:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2023-12-11T19:37:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2023-12-11T19:37:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-12-11T19:37:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap" 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-11T19:37:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-11T19:37:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-11T19:37:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm_log_destination": { 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm_log_destination", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "file", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "file,syslog", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "syslog" 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-11T19:37:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-11T19:37:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2023-12-11T19:37:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2023-12-11T19:37:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-11T19:37:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-11T19:37:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_nvmeof": { 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_nvmeof", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Nvme-of container image", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2023-12-11T19:37:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-11T19:37:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "inventory_list_all": { 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "inventory_list_all", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-11T19:37:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-11T19:37:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.341 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.342 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.343 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.344 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.345 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2023-12-11T19:37:30.346 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.347 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.348 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.349 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-11T19:37:30.350 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.351 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.352 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.354 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2023-12-11T19:37:30.355 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2023-12-11T19:37:30.356 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2023-12-11T19:37:30.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.358 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2023-12-11T19:37:30.359 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.360 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.361 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.362 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.363 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2023-12-11T19:37:30.364 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.365 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.366 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.367 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.368 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.369 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.372 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-11T19:37:30.373 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-11T19:37:30.374 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.375 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.376 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.378 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.379 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.380 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.381 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.382 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.383 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.384 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.385 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.386 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.387 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.389 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-11T19:37:30.390 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-11T19:37:30.391 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.392 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.393 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.394 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "UNSAFE_TLS_v1_2": { 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-11T19:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-11T19:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2023-12-11T19:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-12-11T19:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.427 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.428 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.429 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.430 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.431 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.432 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.435 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2023-12-11T19:37:30.436 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.437 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.438 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2023-12-11T19:37:30.439 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.440 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.441 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.442 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.443 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2023-12-11T19:37:30.444 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2023-12-11T19:37:30.445 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.446 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-11T19:37:30.448 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.449 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.450 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.451 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.452 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.453 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.454 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.455 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2023-12-11T19:37:30.456 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.457 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.459 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.460 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2023-12-11T19:37:30.461 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.462 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.463 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.464 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.465 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.466 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.467 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.468 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.469 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.470 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.471 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-11T19:37:30.473 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.475 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.476 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.477 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.478 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-11T19:37:30.480 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.481 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.482 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2023-12-11T19:37:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2023-12-11T19:37:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2023-12-11T19:37:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.514 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.515 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2023-12-11T19:37:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2023-12-11T19:37:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2023-12-11T19:37:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2023-12-11T19:37:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2023-12-11T19:37:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-12-11T19:37:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.559 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.560 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.561 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.562 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.563 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.564 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2023-12-11T19:37:30.565 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-12-11T19:37:30.566 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.567 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.568 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.569 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.570 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.571 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.572 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.573 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.574 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.575 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.576 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.577 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.578 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.579 INFO:teuthology.orchestra.run.smithi066.stdout: "drive_group_interval": { 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "drive_group_interval", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300.0", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.580 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.581 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.582 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.583 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.584 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.586 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.587 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.588 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.589 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.590 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2023-12-11T19:37:30.591 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.592 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.593 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.594 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.595 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.596 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.597 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.598 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.600 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.601 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.602 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.603 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.604 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.605 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.607 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.609 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.610 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.611 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.612 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.613 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.614 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.615 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.616 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.618 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.619 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2023-12-11T19:37:30.620 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.621 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.622 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.623 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.625 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.626 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2023-12-11T19:37:30.627 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2023-12-11T19:37:30.628 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2023-12-11T19:37:30.629 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2023-12-11T19:37:30.630 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.631 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.632 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.633 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.634 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.635 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.636 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.637 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.638 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.640 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.641 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.642 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.643 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.644 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.645 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.646 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.647 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.648 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.649 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.650 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.651 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-12-11T19:37:30.652 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.653 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.654 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.655 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.657 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "periodic_async_work": { 2023-12-11T19:37:30.658 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "periodic_async_work", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Periodically check for async work", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2023-12-11T19:37:30.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.661 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.662 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.664 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.665 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.666 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.667 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.668 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.669 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.670 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.671 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "dashboard", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "restful" 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.672 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.673 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.674 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.675 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.676 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.677 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.678 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.679 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.680 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.681 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.682 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.684 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.686 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.687 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-11T19:37:30.688 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-11T19:37:30.690 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.691 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.693 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.694 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.695 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.696 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.696 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.696 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.696 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.696 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.697 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.698 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.699 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap" 2023-12-11T19:37:30.700 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-11T19:37:30.701 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-11T19:37:30.702 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-11T19:37:30.703 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.704 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.705 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2023-12-11T19:37:30.706 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.707 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.708 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.709 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.710 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm_log_destination": { 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm_log_destination", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "file", 2023-12-11T19:37:30.711 INFO:teuthology.orchestra.run.smithi066.stdout: "file,syslog", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "syslog" 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.712 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.713 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.714 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.715 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.716 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.717 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.718 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2023-12-11T19:37:30.720 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2023-12-11T19:37:30.721 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.722 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.723 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.724 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_nvmeof": { 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_nvmeof", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.725 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Nvme-of container image", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.726 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-11T19:37:30.727 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.728 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.729 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.730 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-11T19:37:30.731 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.732 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.733 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.735 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "inventory_list_all": { 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "inventory_list_all", 2023-12-11T19:37:30.736 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.737 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.738 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-11T19:37:30.739 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.740 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.741 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.742 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.743 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.744 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.745 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.746 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.747 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2023-12-11T19:37:30.748 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.749 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-11T19:37:30.750 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.751 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.752 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.753 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.754 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.755 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2023-12-11T19:37:30.756 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2023-12-11T19:37:30.757 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2023-12-11T19:37:30.758 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.759 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.760 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.761 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.762 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.763 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.764 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.765 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.766 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.767 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.768 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2023-12-11T19:37:30.769 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-11T19:37:30.770 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2023-12-11T19:37:30.771 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.772 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.773 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.774 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.775 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.776 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.777 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.778 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.779 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.781 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.782 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2023-12-11T19:37:30.783 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-11T19:37:30.784 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.785 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.786 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.787 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.788 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.789 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.790 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.791 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.792 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.793 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.794 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-11T19:37:30.795 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-11T19:37:30.796 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-11T19:37:30.797 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.798 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.799 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.800 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2023-12-11T19:37:30.801 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.802 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.803 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.804 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.805 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "UNSAFE_TLS_v1_2": { 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.806 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.807 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.808 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.810 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2023-12-11T19:37:30.811 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2023-12-11T19:37:30.812 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.813 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.814 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.815 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.817 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.818 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.819 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.820 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.821 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.822 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.824 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.827 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.828 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.829 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.830 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.831 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.832 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.835 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2023-12-11T19:37:30.836 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.837 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.838 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.839 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.840 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.841 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.842 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.843 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.844 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.845 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.846 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.847 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2023-12-11T19:37:30.848 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.850 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:30.851 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.852 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.853 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.854 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.855 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.856 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.857 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.858 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.859 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.860 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.863 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.864 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.865 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.866 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.867 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.868 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.869 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.870 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.871 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.872 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.873 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2023-12-11T19:37:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2023-12-11T19:37:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2023-12-11T19:37:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2023-12-11T19:37:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2023-12-11T19:37:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2023-12-11T19:37:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2023-12-11T19:37:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2023-12-11T19:37:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2023-12-11T19:37:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-12-11T19:37:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-12-11T19:37:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-12-11T19:37:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.977 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.979 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "drive_group_interval": { 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "drive_group_interval", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300.0", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.980 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.981 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.982 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.983 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.984 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2023-12-11T19:37:30.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:30.986 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.987 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.988 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.989 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:30.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.991 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.992 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.994 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.995 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.997 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:30.999 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.000 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.003 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.004 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.005 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.006 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.008 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.009 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.011 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.012 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.013 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.014 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.015 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.016 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.018 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.019 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.020 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.021 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.022 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.023 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.024 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.025 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.026 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.027 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.028 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.029 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.031 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2023-12-11T19:37:31.032 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2023-12-11T19:37:31.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.035 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.036 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.039 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.040 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.041 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.042 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.043 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.044 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.045 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.047 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.048 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.049 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.051 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.052 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.056 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.057 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.058 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "periodic_async_work": { 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "periodic_async_work", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Periodically check for async work", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-11T19:37:31.060 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.061 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.062 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.063 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-12-11T19:37:31.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.067 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.068 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.069 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2023-12-11T19:37:31.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.071 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "services": { 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "dashboard": "https://172.21.15.66:8443/" 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "always_on_modules": { 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "octopus": [ 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-12-11T19:37:31.072 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "pacific": [ 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "quincy": [ 2023-12-11T19:37:31.073 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "reef": [ 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-12-11T19:37:31.074 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "last_failure_osd_epoch": 3, 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "active_clients": [ 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.075 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 188604032 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "libcephsqlite", 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 980393415 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3400214772 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-12-11T19:37:31.077 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-12-11T19:37:31.078 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2264173503 2023-12-11T19:37:31.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.078 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:31.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:31.078 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-12-11T19:37:31.078 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:37:31.079 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:29 smithi070 ceph-mon[104044]: pgmap v477: 97 pgs: 97 active+clean; 3.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 161 op/s 2023-12-11T19:37:31.079 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:29 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1612470368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:37:31.079 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:29 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3494031452' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-11T19:37:31.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:29 smithi167 ceph-mon[102750]: pgmap v477: 97 pgs: 97 active+clean; 3.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 161 op/s 2023-12-11T19:37:31.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:29 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1612470368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:37:31.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:29 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3494031452' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-11T19:37:31.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.852+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01500a1500 msgr2=0x7f01500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:31.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01500a1500 0x7f01500a39c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01681024f0 tx=0x7f0158009260 comp rx=0 tx=0).stop 2023-12-11T19:37:31.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 msgr2=0x7f0168199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:37:31.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 0x7f0168199af0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c9b0 tx=0x7f015c00ccc0 comp rx=0 tx=0).stop 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 shutdown_connections 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f01500a1500 0x7f01500a39c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01681014a0 0x7f01681995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0168103030 0x7f01681a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 --2- 172.21.15.66:0/3494031452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0168102690 0x7f0168199af0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 >> 172.21.15.66:0/3494031452 conn(0x7f01680fca10 msgr2=0x7f0168073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 shutdown_connections 2023-12-11T19:37:31.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:37:29.853+0000 7f016e5ff700 1 -- 172.21.15.66:0/3494031452 wait complete. 2023-12-11T19:37:31.087 DEBUG:tasks.check_counter:Getting stats from a 2023-12-11T19:37:31.087 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-11T19:37:31.087 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:29 smithi066 ceph-mon[97920]: pgmap v477: 97 pgs: 97 active+clean; 3.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 161 op/s 2023-12-11T19:37:31.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:29 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1612470368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:37:31.088 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:29 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3494031452' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-11T19:37:31.891 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:37:32.219 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:31 smithi066 ceph-mon[97920]: pgmap v478: 97 pgs: 97 active+clean; 997 KiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.8 KiB/s wr, 228 op/s 2023-12-11T19:37:32.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:31 smithi167 ceph-mon[102750]: pgmap v478: 97 pgs: 97 active+clean; 997 KiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.8 KiB/s wr, 228 op/s 2023-12-11T19:37:32.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:31 smithi070 ceph-mon[104044]: pgmap v478: 97 pgs: 97 active+clean; 997 KiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.8 KiB/s wr, 228 op/s 2023-12-11T19:37:33.552 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:37:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:33.552 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 352, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 352, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 119443, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 108677, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 2, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.045300128, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.014587128, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.011480878, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.011547623, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 351, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.022521126, 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000064162 2023-12-11T19:37:33.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 242, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 315, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 60098, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 280011, 2023-12-11T19:37:33.554 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.033769884, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.013080413, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.007902690, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.008938381, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 313, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.018380137, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000058722 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.555 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 204992 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 435, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 427, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 565226, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 158940, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.053144959, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.018748277, 2023-12-11T19:37:33.556 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.022595397, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.010657298, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 426, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.023820192, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000055915 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 479408, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 58176 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.557 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1549, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.045535116, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000029396 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2023-12-11T19:37:33.558 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.559 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2023-12-11T19:37:33.560 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 1, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2023-12-11T19:37:33.561 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:37:33.562 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2023-12-11T19:37:33.563 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2023-12-11T19:37:33.564 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 3, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2023-12-11T19:37:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4195390, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 4194892, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.566 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 3 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 3, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 3, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 0, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 1, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 1, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 0, 2023-12-11T19:37:33.567 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 47232, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207228 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2023-12-11T19:37:33.568 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.571 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.576 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.578 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 1, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1923, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-12-11T19:37:33.581 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:33.582 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:33.583 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-12-11T19:37:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:33.584 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 197376, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 54, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 160, 2023-12-11T19:37:33.585 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 8320, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 13, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2023-12-11T19:37:33.586 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 913, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 913, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 913, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.610995380, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000669217 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 913, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 914 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 913, 2023-12-11T19:37:33.587 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 913, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 572, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 339, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2023-12-11T19:37:33.588 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:33.589 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2023-12-11T19:37:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2023-12-11T19:37:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2023-12-11T19:37:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2023-12-11T19:37:33.590 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 88, 2023-12-11T19:37:33.591 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 6, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 6, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2023-12-11T19:37:33.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:33.593 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 1025, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 643943, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 1025, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1025, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 643943, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.594 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 913, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 1091, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 913, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1091, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.595 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 913, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 913, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 913, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 913, 2023-12-11T19:37:33.596 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:33.597 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55b6f1e76a00": { 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:33.598 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:33.599 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:37:34.006 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-12-11T19:37:34.006 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-12-11T19:37:34.006 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-12-11T19:37:34.006 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-12-11T19:37:34.006 DEBUG:tasks.check_counter:Getting stats from d 2023-12-11T19:37:34.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-11T19:37:34.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:33 smithi167 ceph-mon[102750]: pgmap v479: 97 pgs: 97 active+clean; 997 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 224 op/s 2023-12-11T19:37:34.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:33 smithi066 ceph-mon[97920]: pgmap v479: 97 pgs: 97 active+clean; 997 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 224 op/s 2023-12-11T19:37:34.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:33 smithi070 ceph-mon[104044]: pgmap v479: 97 pgs: 97 active+clean; 997 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 224 op/s 2023-12-11T19:37:34.807 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:37:36.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:35 smithi167 ceph-mon[102750]: pgmap v480: 97 pgs: 97 active+clean; 997 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 225 op/s 2023-12-11T19:37:36.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:35 smithi066 ceph-mon[97920]: pgmap v480: 97 pgs: 97 active+clean; 997 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 225 op/s 2023-12-11T19:37:36.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:35 smithi070 ceph-mon[104044]: pgmap v480: 97 pgs: 97 active+clean; 997 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 225 op/s 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2023-12-11T19:37:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.003672352, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000736027, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:37:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 107, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 104, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 440688, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 61696, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.022663272, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.008493302, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.014637180, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.002178905, 2023-12-11T19:37:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 103, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.006739822, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000065435 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 440688, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 61696 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 5, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 81, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3120, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 91120, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 3, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 2, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.008969184, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.006009355, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001841544, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 80, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.005624392, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000070304 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3120, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 91120 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 148903, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 29, 2023-12-11T19:37:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2023-12-11T19:37:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 108, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 425594, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 108, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 108, 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 425594, 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:36.382 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:37:36.820 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-12-11T19:37:36.820 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-12-11T19:37:36.820 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-12-11T19:37:36.820 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-12-11T19:37:36.820 DEBUG:tasks.check_counter:Getting stats from g 2023-12-11T19:37:36.820 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-11T19:37:37.618 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:37:38.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:37 smithi066 ceph-mon[97920]: pgmap v481: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 222 op/s 2023-12-11T19:37:38.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:37 smithi167 ceph-mon[102750]: pgmap v481: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 222 op/s 2023-12-11T19:37:38.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:37 smithi070 ceph-mon[104044]: pgmap v481: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 222 op/s 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 452, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 445, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 594192, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 165164, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.068362779, 2023-12-11T19:37:39.209 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.027681083, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.022722672, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.012557343, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 444, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.028716727, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000064677 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 479952, 2023-12-11T19:37:39.210 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 59424 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 600, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 674, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 196700, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 393098, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.059654754, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.021125336, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.015649841, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.017559545, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 672, 2023-12-11T19:37:39.211 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.030191463, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000044927 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 206752 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 8, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 8, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 4589, 2023-12-11T19:37:39.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 1704, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.003299407, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000205129, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002895592, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000097117, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 7, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.005166789, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000738112 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.213 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1631, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.047291468, 2023-12-11T19:37:39.214 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000028995 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2023-12-11T19:37:39.215 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 2, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2023-12-11T19:37:39.216 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 2, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 2, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 1, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2023-12-11T19:37:39.217 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2023-12-11T19:37:39.218 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:39.219 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:39.220 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2023-12-11T19:37:39.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 33, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2023-12-11T19:37:39.222 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4237986, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 4232264, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 33 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 4, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 12, 2023-12-11T19:37:39.223 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 8, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 2, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 9, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 7, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 2, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 10, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 8, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 47360, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 207228 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2023-12-11T19:37:39.224 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:39.226 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:39.227 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2023-12-11T19:37:39.228 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.229 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.230 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.233 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.234 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 1, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1925, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:39.236 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:39.237 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:39.238 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 197336, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 53, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:39.239 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 160, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 13200, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 36, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2023-12-11T19:37:39.240 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 943, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 943, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 943, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.614995361, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000652168 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 943, 2023-12-11T19:37:39.241 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 944 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 943, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 943, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 584, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 357, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2023-12-11T19:37:39.242 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2023-12-11T19:37:39.243 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2023-12-11T19:37:39.244 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 88, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 6, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 6, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2023-12-11T19:37:39.245 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:37:39.246 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 1056, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 691763, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 1056, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:39.247 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1056, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 691763, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:39.248 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 943, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 43687, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 943, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 43687, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:39.249 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 943, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 943, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 943, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 943, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:39.250 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.251 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55f26fefea00": { 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:39.252 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:39.253 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:39.253 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.253 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:39.253 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:37:39.734 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-12-11T19:37:39.734 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-12-11T19:37:39.734 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-12-11T19:37:39.734 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-12-11T19:37:39.734 DEBUG:tasks.check_counter:Getting stats from j 2023-12-11T19:37:39.735 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-11T19:37:40.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:39 smithi167 ceph-mon[102750]: pgmap v482: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 129 op/s 2023-12-11T19:37:40.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:39 smithi066 ceph-mon[97920]: pgmap v482: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 129 op/s 2023-12-11T19:37:40.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:39 smithi070 ceph-mon[104044]: pgmap v482: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 129 op/s 2023-12-11T19:37:40.532 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2023-12-11T19:37:42.211 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.003527543, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000775675, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.212 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 5, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 82, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3120, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 91296, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 3, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 2, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.010252861, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.006464926, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002673666, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:42.213 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 81, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.006091038, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000075198 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3120, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 91296 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 109, 2023-12-11T19:37:42.214 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 106, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 441232, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 62944, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.025689504, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.009841291, 2023-12-11T19:37:42.215 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.017660445, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.002296201, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 105, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.007014821, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000066807 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 441232, 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 62944 2023-12-11T19:37:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:42.217 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:42.218 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 148823, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 27, 2023-12-11T19:37:42.219 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2023-12-11T19:37:42.220 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 110, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 425854, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 110, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 110, 2023-12-11T19:37:42.221 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 425854, 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-12-11T19:37:42.222 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-12-11T19:37:42.232 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:41 smithi066 ceph-mon[97920]: pgmap v483: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 76 op/s 2023-12-11T19:37:42.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:41 smithi167 ceph-mon[102750]: pgmap v483: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 76 op/s 2023-12-11T19:37:42.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:41 smithi070 ceph-mon[104044]: pgmap v483: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 76 op/s 2023-12-11T19:37:42.853 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-11T19:37:42.853 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-11T19:37:42.853 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-12-11T19:37:42.853 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-12-11T19:37:42.853 DEBUG:tasks.check_counter:Getting stats from b 2023-12-11T19:37:42.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-11T19:37:43.648 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:37:44.123 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:43 smithi070 ceph-mon[104044]: pgmap v484: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-12-11T19:37:44.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:43 smithi167 ceph-mon[102750]: pgmap v484: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-12-11T19:37:44.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:43 smithi066 ceph-mon[97920]: pgmap v484: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-12-11T19:37:45.006 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-12-11T19:37:45.006 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:45.006 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1879, 2023-12-11T19:37:45.006 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 1850, 2023-12-11T19:37:45.006 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 604488, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 1065660, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 10, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 3, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.212421806, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.073539442, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.062564564, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.053179761, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1854, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.250436126, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000135078 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1077, 2023-12-11T19:37:45.007 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000329014, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000305 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1862, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 1898, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 593868, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 841173, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 9, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 5, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.206523208, 2023-12-11T19:37:45.008 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.074849657, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.063822476, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.051629037, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1900, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.311421448, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000163906 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1097, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000296697, 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000270 2023-12-11T19:37:45.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 181328 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 1825, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 1554, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1008312, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 1206488, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 80, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 10, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.237987722, 2023-12-11T19:37:45.010 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.068923460, 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.097879240, 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.062163072, 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1544, 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.406080471, 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000263005 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.011 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 676, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000139556, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000206 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 574144, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 99328 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:45.012 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 527, 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.133315101, 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000252969 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:45.013 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2569, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.108589884, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000042269 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 47, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 46, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 46, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.073707573, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001602338 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2023-12-11T19:37:45.014 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 1, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 97, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 23, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 4, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 19, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 0, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 16, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 1, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 2, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 2, 2023-12-11T19:37:45.015 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 2, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 73, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 60, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 1, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 0, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 187, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 1, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 1, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 1, 2023-12-11T19:37:45.016 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 3, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 49, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 4, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 246, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 0, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 49, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 33, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 4, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 29, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 16, 2023-12-11T19:37:45.017 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 4, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 15, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 47, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 60, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 39, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 4, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 28 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 10, 2023-12-11T19:37:45.018 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 28, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 4, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 4, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:45.019 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 4, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 1, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "evlrg": 0, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 122, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 114, 2023-12-11T19:37:45.020 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 114, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 8, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 2, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 18, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 17, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 17, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 1, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "segmjr": 1, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 1, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 4553081, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 4579999, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 0, 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2023-12-11T19:37:45.021 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 105, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.252421993, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.002404018 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 0 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 23, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 32, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 9, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 18, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 20, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 2, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 23, 2023-12-11T19:37:45.022 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 33, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 10, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 2, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 18, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 16, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 52780, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 207228 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 75, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 2, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 47, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 12, 2023-12-11T19:37:45.023 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 4, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 6, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.005219615, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000869935 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 16, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.005142659, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000321416 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 8, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.014500327, 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001812540 2023-12-11T19:37:45.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.029 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2023-12-11T19:37:45.029 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1, 2023-12-11T19:37:45.029 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000528512, 2023-12-11T19:37:45.030 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000528512 2023-12-11T19:37:45.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.030 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.006515252, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.003257626 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.002168574, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001084287 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.031 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.013265070, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.006632535 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.032 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:45.033 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.033 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.033 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.033 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2023-12-11T19:37:45.033 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1, 2023-12-11T19:37:45.034 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.001043503, 2023-12-11T19:37:45.034 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001043503 2023-12-11T19:37:45.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.034 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:45.034 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.034 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2023-12-11T19:37:45.035 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 6, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.019163765, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.003193960 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.006160296, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.003080148 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.036 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 1, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 2, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 1, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 1, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 11, 2023-12-11T19:37:45.037 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 11, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 245, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:45.038 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:45.039 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 291049, 2023-12-11T19:37:45.040 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 114, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 176, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 2, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 160, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-12-11T19:37:45.041 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 95274, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 699, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 3735, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 491271, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 31, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 3735, 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2023-12-11T19:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3735, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 36.106935553, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.009667184 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 3735, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 3981 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 3735, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 61, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 3674, 2023-12-11T19:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 77, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 29, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 8, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 109, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 65, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 3330, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:45.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 53, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 58, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2023-12-11T19:37:45.045 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2023-12-11T19:37:45.046 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 88, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 3, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 12, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 12, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 236, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 0, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 16 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 1, 2023-12-11T19:37:45.047 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 2560, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 1, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed_ops": 2818, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 4, 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:45.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 5562, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1689570, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 5562, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 5562, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1689570, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.049 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 3735, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 491675, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 3735, 2023-12-11T19:37:45.050 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 491675, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:45.051 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 3735, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 3735, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 3735, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 3735, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.052 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 4, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 404, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 4, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 4, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 404, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.053 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x559f66e7a6e0": { 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 122, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 385695, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 122, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 105, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 385695, 2023-12-11T19:37:45.054 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:45.055 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:45.055 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:45.055 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:45.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:45.055 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-12-11T19:37:45.429 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2023-12-11T19:37:45.429 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2023-12-11T19:37:45.429 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=10 2023-12-11T19:37:45.430 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-12-11T19:37:45.430 DEBUG:tasks.check_counter:Getting stats from e 2023-12-11T19:37:45.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-11T19:37:46.200 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:37:46.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:45 smithi167 ceph-mon[102750]: pgmap v485: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2023-12-11T19:37:46.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:45 smithi066 ceph-mon[97920]: pgmap v485: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2023-12-11T19:37:46.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:45 smithi070 ceph-mon[104044]: pgmap v485: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 11 op/s 2023-12-11T19:37:47.637 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-12-11T19:37:47.637 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 207, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 194, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 551945, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 153796, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 14, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 2, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.040974399, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.011530348, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.025285202, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.002769086, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 194, 2023-12-11T19:37:47.638 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.035301543, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000181966 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 74, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000016537, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000223 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 507216, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 68416 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 47, 2023-12-11T19:37:47.639 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 126, 2023-12-11T19:37:47.640 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 43306, 2023-12-11T19:37:47.640 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 225845, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 9, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 5, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.013929732, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.005254220, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.007633374, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000397769, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 124, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.036684842, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000295845 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 55, 2023-12-11T19:37:47.641 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000012862, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000233 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 179840 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 322, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 338, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 151274, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 251683, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 16, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 7, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.055821633, 2023-12-11T19:37:47.642 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.025910369, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.028527683, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.001877259, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 336, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.048682281, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000144887 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 599, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000248186, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000414 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:37:47.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 48, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.001609480, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000033530 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:47.644 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000019260, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000009630 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 1, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 0, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.645 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 1, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 11, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 16, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 16, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 13, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 4, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 0, 2023-12-11T19:37:47.646 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 3, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 8, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 4, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 1, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 3, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 0, 2023-12-11T19:37:47.647 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 5, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 1073742070, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:37:47.648 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 0 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 0, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 11, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 7, 2023-12-11T19:37:47.649 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 9, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:47.650 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2023-12-11T19:37:47.651 INFO:teuthology.orchestra.run.smithi070.stdout: "evlrg": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 3, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 3, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 2, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 2, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "segmjr": 2, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 0, 2023-12-11T19:37:47.652 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 4194304, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 4195390, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 0, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 2, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.004649159, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.002324579 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 0 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 20, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 22, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 2, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 19, 2023-12-11T19:37:47.653 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 22, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 3, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 16, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 20, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 4, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 0, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 0, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 0, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 49536, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 199036 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 1, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 0, 2023-12-11T19:37:47.654 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 1, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 5, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 0, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.655 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.656 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.657 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.658 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2023-12-11T19:37:47.659 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.660 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.661 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2023-12-11T19:37:47.661 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.661 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.661 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.662 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2023-12-11T19:37:47.663 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2023-12-11T19:37:47.664 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.665 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2023-12-11T19:37:47.666 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 1, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 1, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 1, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 247, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:47.667 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:47.668 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-12-11T19:37:47.669 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 262665, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 91, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:47.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 160, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 79645, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 322, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2023-12-11T19:37:47.671 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 37, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 11298, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 27, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 37, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 37, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.262992241, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.007107898 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 37, 2023-12-11T19:37:47.672 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 59 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 37, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 37, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 10, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 2, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 6, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2023-12-11T19:37:47.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 6, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 20, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:47.674 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2023-12-11T19:37:47.675 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 88, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 2, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 9, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 9, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 14, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 0, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 1 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.676 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 1, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 0, 2023-12-11T19:37:47.677 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed_ops": 0, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 0, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 572, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 684654, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 572, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:47.678 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 572, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 684654, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:47.679 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 37, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 11298, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 37, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 11298, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:47.680 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 37, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 37, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 37, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 37, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:47.682 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x55cd92dc46e0": { 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 3, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1086, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 3, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:47.683 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 2, 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1086, 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:47.684 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-12-11T19:37:48.089 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-12-11T19:37:48.089 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-12-11T19:37:48.089 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-12-11T19:37:48.089 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=11 2023-12-11T19:37:48.089 DEBUG:tasks.check_counter:Getting stats from h 2023-12-11T19:37:48.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-11T19:37:48.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:47 smithi167 ceph-mon[102750]: pgmap v486: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2023-12-11T19:37:48.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:47 smithi066 ceph-mon[97920]: pgmap v486: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2023-12-11T19:37:48.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:47 smithi070 ceph-mon[104044]: pgmap v486: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2023-12-11T19:37:48.862 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:37:50.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:49 smithi167 ceph-mon[102750]: pgmap v487: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 897, 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 883, 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1123583, 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 302825, 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 5, 2023-12-11T19:37:50.271 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 1, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.108510494, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.044745082, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.033350947, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.021770215, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 882, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.045351059, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000051418 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 510112, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 63040 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 5, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 83, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 1816, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 182558, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 5, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 4, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.007527057, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.004006086, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.002193204, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.000118307, 2023-12-11T19:37:50.273 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 81, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.010158183, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000125409 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 181248 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 341, 2023-12-11T19:37:50.274 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 341, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 85968, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 105262, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 4, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 3, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.037360229, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.010735770, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.010176080, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.012541412, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 340, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.017592374, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000051742 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1879, 2023-12-11T19:37:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.054967645, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000029253 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 0, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 0, 2023-12-11T19:37:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2023-12-11T19:37:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 19, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 5, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 14, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 11, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 2, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 0, 2023-12-11T19:37:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 0, 2023-12-11T19:37:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 4, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 246, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:37:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 0 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 0, 2023-12-11T19:37:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:37:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "evlrg": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 0, 2023-12-11T19:37:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 122, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 2, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "segmjr": 18, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 4553081, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 4579999, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 4579400, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 122 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 15, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 20, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 5, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 10, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 12, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 2, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 19, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 25, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 6, 2023-12-11T19:37:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 48256, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 207228 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2023-12-11T19:37:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 1, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 2, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 1, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 1, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 0, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 0, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 2005, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:50.298 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:50.299 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 189256, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 49, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 160, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 58770, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 276, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2023-12-11T19:37:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 1112, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 26, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 4, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 1112, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1112, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.548983809, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000493690 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1112, 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 1113 2023-12-11T19:37:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 1112, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 1112, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 688, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 421, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 0, 2023-12-11T19:37:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 2, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2023-12-11T19:37:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2023-12-11T19:37:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 88, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 2, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 5, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 5, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2023-12-11T19:37:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 0, 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 2, 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 0 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 0, 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2023-12-11T19:37:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 0, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed_ops": 0, 2023-12-11T19:37:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 0, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 1239, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 1090479, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 1239, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 1239, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1090479, 2023-12-11T19:37:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 1112, 2023-12-11T19:37:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 385705, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 1112, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 385705, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 1112, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 1112, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 1112, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 1112, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2023-12-11T19:37:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x5594c773aa00": { 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 0, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 0, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:50.318 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-12-11T19:37:50.319 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:49 smithi066 ceph-mon[97920]: pgmap v487: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:37:50.319 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:49 smithi070 ceph-mon[104044]: pgmap v487: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-11T19:37:50.754 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-12-11T19:37:50.754 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-12-11T19:37:50.754 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-12-11T19:37:50.754 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-12-11T19:37:50.755 DEBUG:tasks.check_counter:Getting stats from k 2023-12-11T19:37:50.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-11T19:37:51.524 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T19:37:52.037 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:51 smithi070 ceph-mon[104044]: pgmap v488: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-11T19:37:52.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:51 smithi167 ceph-mon[102750]: pgmap v488: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-11T19:37:52.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:51 smithi066 ceph-mon[97920]: pgmap v488: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 448, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 480, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 181993, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 257104, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 11, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 5, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.069700211, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.031630752, 2023-12-11T19:37:52.919 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.028066638, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.005542741, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 478, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.058242058, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000121845 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 617, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000182268, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000295 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:37:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 336, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 346, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 579038, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 212150, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 14, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 6, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.051992880, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.017616250, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.033433653, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.005578691, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 346, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.043374689, 2023-12-11T19:37:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000125360 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 150, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000045250, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000301 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 500368, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 73360 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_messages": 142, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages": 224, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_bytes": 66811, 2023-12-11T19:37:52.922 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_bytes": 334733, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_created_connections": 17, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_active_connections": 9, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_total_time": 0.026309216, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_send_time": 0.008539760, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_recv_time": 0.012723218, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_running_fast_dispatch_time": 0.003078748, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 222, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.064086800, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000288679 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 69, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000016304, 2023-12-11T19:37:52.923 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000236 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "msgr_send_encrypted_bytes": 182000 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "cct": { 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "total_workers": 1, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "unhealthy_workers": 0 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-MDSRank": { 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 109, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.009235407, 2023-12-11T19:37:52.924 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000084728 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "queue_len": 0, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "complete_latency": { 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 245, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.010346868, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000042232 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "request": 7, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "reply": 6, 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "reply_latency": { 2023-12-11T19:37:52.925 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 6, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.005487580, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000914596 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_reply": 0, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "forward": 1, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_commit": 11, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_split": 0, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_merge": 0, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes": 16, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_top": 2, 2023-12-11T19:37:52.926 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_bottom": 1, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pin_tail": 13, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_pinned": 13, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_expired": 8, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "inodes_with_caps": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "subtrees": 3, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse": 14, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_hit": 5, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_forward": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_discover": 2, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "traverse_lock": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "load_cent": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "q": 0, 2023-12-11T19:37:52.927 INFO:teuthology.orchestra.run.smithi070.stdout: "exported": 1, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "exported_inodes": 3, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "imported": 1, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "imported_inodes": 1, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_backtrace_fetch": 1, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rfiles": 5, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rbytes": 1073742070, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "root_rsnaps": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_fetch": 1, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:37:52.928 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "scrub_file_inodes": 1, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_revoke": 4, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_grant": 22, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps": 26, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_caps_dirty": 20, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_cap_release": 2, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "process_request_cap_release": 4 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache": { 2023-12-11T19:37:52.929 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update": 3, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_update_receipt": 3, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_try_discover": 2, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_send_discover": 4, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "dir_handle_discover": 3, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_created": 1, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_enqueued": 1, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "strays_migrated": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:52.930 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_started": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_completed": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_exportdir": 1, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_flush": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ireq_inodestats": 0 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log": { 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "evlrg": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "evadd": 33, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "evex": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "evtrm": 0, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "ev": 33, 2023-12-11T19:37:52.931 INFO:teuthology.orchestra.run.smithi070.stdout: "evexg": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "evexd": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "segadd": 2, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "segex": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "segtrm": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "seg": 2, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "segmjr": 2, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "segexg": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "segexd": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "expos": 4194304, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "wrpos": 4237986, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "rdpos": 0, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "jlat": { 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 28, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.062931383, 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.002247549 2023-12-11T19:37:52.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "replayed": 0 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mem": { 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "ino": 20, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "ino+": 24, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "ino-": 4, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "dir": 19, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "dir+": 22, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "dir-": 3, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "dn": 16, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "dn+": 25, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "dn-": 9, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "cap": 0, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "cap+": 3, 2023-12-11T19:37:52.933 INFO:teuthology.orchestra.run.smithi070.stdout: "cap-": 3, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "rss": 50304, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "heap": 199036 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server": { 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_client_request": 7, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "dispatch_server_request": 0, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_request": 7, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_client_session": 6, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "handle_peer_request": 1, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "req_create_latency": { 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.003101910, 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.003101910 2023-12-11T19:37:52.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getattr_latency": { 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "req_link_latency": { 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookup_latency": { 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 4, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.001166976, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000291744 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupino_latency": { 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupname_latency": { 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.937 INFO:teuthology.orchestra.run.smithi070.stdout: "req_lssnap_latency": { 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mkdir_latency": { 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mknod_latency": { 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "req_mksnap_latency": { 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "req_open_latency": { 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "req_readdir_latency": { 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.939 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rename_latency": { 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.940 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmdir_latency": { 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.941 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setattr_latency": { 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setlayout_latency": { 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "req_setxattr_latency": { 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "req_symlink_latency": { 2023-12-11T19:37:52.943 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "req_unlink_latency": { 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 1, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.001218694, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.001218694 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.944 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessions": { 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "session_count": 1, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "session_add": 1, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "session_remove": 0, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_open": 1, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "sessions_stale": 0, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "total_load": 5, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "average_load": 5, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "avg_session_uptime": 253, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.945 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool": { 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:52.946 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:52.947 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_items": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_bytes": 289017, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_anon_items": 111, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_items": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:52.948 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_items": 160, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "pgmap_items": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_bytes": 79693, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_co_items": 328, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_1_items": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: "unittest_2_items": 0 2023-12-11T19:37:52.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter": { 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_active": 0, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_laggy": 0, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send": 339, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_send_bytes": 55529, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_resend": 27, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_reply": 339, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_latency": { 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 339, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 1.935942964, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.005710746 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.950 INFO:teuthology.orchestra.run.smithi070.stdout: "op_inflight": 0, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "oplen_avg": { 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 339, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 367 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "op": 339, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "op_r": 2, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "op_w": 337, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "op_rmw": 0, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "op_pg": 0, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_stat": 0, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_create": 10, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_read": 1, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_write": 29, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writefull": 11, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_writesame": 0, 2023-12-11T19:37:52.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_append": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_zero": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_truncate": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_delete": 268, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_mapext": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_getxattr": 1, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_setxattr": 20, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_call": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_watch": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_notify": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:52.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osdop_other": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_active": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_send": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_resend": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "linger_ping": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_active": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_send": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "poolop_resend": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_active": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_send": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "poolstat_resend": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_active": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_send": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "statfs_resend": 0, 2023-12-11T19:37:52.953 INFO:teuthology.orchestra.run.smithi070.stdout: "command_active": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "command_send": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "command_resend": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "map_epoch": 88, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "map_full": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "map_inc": 2, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_sessions": 12, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_open": 12, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_session_close": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_laggy": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_wr": 23, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_rd": 0, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_del": 4 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "oft": { 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_objs": 1, 2023-12-11T19:37:52.954 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_updates": 0, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "omap_total_removes": 0 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "osd-slow-ops": { 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "slow_ops_count": 0 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "purge_queue": { 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_ops_high_water": 256, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing": 0, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executing_high_water": 1, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed_ops": 256, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_executed": 1, 2023-12-11T19:37:52.955 INFO:teuthology.orchestra.run.smithi070.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 922, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 734281, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 922, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 922, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 734281, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.956 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 104857600, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 339, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 55630, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 339, 2023-12-11T19:37:52.957 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 55630, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 1024, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 0, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 0, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:52.958 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 339, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 339, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 339, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 339, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 1, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 101, 2023-12-11T19:37:52.959 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 1, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 1, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 101, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "throttle-write_buf_throttle-0x559adb32c6e0": { 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "val": 0, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "max": 3758096384, 2023-12-11T19:37:52.960 INFO:teuthology.orchestra.run.smithi070.stdout: "get_started": 0, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "get": 33, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "get_sum": 43682, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "get_or_fail_success": 33, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "take": 0, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "take_sum": 0, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "put": 28, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "put_sum": 43682, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "wait": { 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "avgcount": 0, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "sum": 0.000000000, 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: "avgtime": 0.000000000 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-12-11T19:37:52.961 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-12-11T19:37:53.316 INFO:tasks.check_counter:Daemon mds.k mds.exported=1 2023-12-11T19:37:53.316 INFO:tasks.check_counter:Daemon mds.k mds.imported=1 2023-12-11T19:37:53.316 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=3 2023-12-11T19:37:53.316 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=3 2023-12-11T19:37:53.316 DEBUG:tasks.check_counter:Getting stats from c 2023-12-11T19:37:53.316 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-11T19:37:54.119 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:37:54.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:53 smithi167 ceph-mon[102750]: pgmap v489: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:37:54.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:53 smithi066 ceph-mon[97920]: pgmap v489: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:37:54.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:53 smithi070 ceph-mon[104044]: pgmap v489: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:37:56.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:55 smithi167 ceph-mon[102750]: pgmap v490: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:37:56.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:55 smithi066 ceph-mon[97920]: pgmap v490: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:37:56.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:55 smithi070 ceph-mon[104044]: pgmap v490: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:37:57.326 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1016, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1017, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 290637, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 315672, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.107003951, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.036252599, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.026701988, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.031844812, 2023-12-11T19:37:57.327 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1015, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.049483272, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000048751 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 133, 2023-12-11T19:37:57.328 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 117, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 514012, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 65276, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 3, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.026688052, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.009391727, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.017573512, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.002654862, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 117, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.011337506, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000096901 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.329 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 513456, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 64592 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 82, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 428, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 183495, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2023-12-11T19:37:57.330 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.007440193, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.003712574, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.002390659, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000017648, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 80, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.007503318, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000093791 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:37:57.331 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 183184 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1742, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.049976388, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000028689 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.332 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.333 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 0, 2023-12-11T19:37:57.334 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2023-12-11T19:37:57.335 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2023-12-11T19:37:57.335 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2023-12-11T19:37:57.335 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 1, 2023-12-11T19:37:57.335 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2023-12-11T19:37:57.335 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2023-12-11T19:37:57.335 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2023-12-11T19:37:57.336 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:37:57.337 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2023-12-11T19:37:57.338 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2023-12-11T19:37:57.339 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 5, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2023-12-11T19:37:57.340 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195920, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 4195422, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 5 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.341 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 3, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 3, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 1, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 1, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 47360, 2023-12-11T19:37:57.342 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207228 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2023-12-11T19:37:57.343 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.344 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2023-12-11T19:37:57.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2023-12-11T19:37:57.346 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.347 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.348 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.349 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.351 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2023-12-11T19:37:57.352 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.353 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1986, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-12-11T19:37:57.354 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:37:57.355 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:37:57.356 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 181199, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 45, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-12-11T19:37:57.357 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7624, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 160, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 8320, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 13, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-12-11T19:37:57.358 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1018, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1018, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1018, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.647988632, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000636531 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.359 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1018, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1019 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1018, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 1018, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 648, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 368, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2023-12-11T19:37:57.360 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2023-12-11T19:37:57.361 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2023-12-11T19:37:57.362 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 88, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 4, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 4, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2023-12-11T19:37:57.363 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2023-12-11T19:37:57.364 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 1147, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 692411, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 1147, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1147, 2023-12-11T19:37:57.365 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 692411, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:57.366 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1018, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1621, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1018, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1621, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.367 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1018, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1018, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1018, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1018, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.368 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:37:57.369 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55d1f7198a00": { 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:37:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:37:57.371 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-12-11T19:37:57.818 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-12-11T19:37:57.818 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-12-11T19:37:57.818 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-12-11T19:37:57.818 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-12-11T19:37:57.818 DEBUG:tasks.check_counter:Getting stats from f 2023-12-11T19:37:57.819 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.f perf dump 2023-12-11T19:37:58.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:37:57 smithi167 ceph-mon[102750]: pgmap v491: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-11T19:37:58.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:57 smithi066 ceph-mon[97920]: pgmap v491: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-11T19:37:58.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:37:57 smithi070 ceph-mon[104044]: pgmap v491: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-11T19:37:58.595 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:38:00.118 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 76, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 76, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 52595, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 73723, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.017460963, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.002639955, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.009369495, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000583414, 2023-12-11T19:38:00.119 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 75, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.027780569, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000370407 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 90, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000056432, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000627 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 4896, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 2032 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:38:00.120 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 212, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 198, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 563677, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 218265, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 17, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 8, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.041942968, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.013879075, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.029004167, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.003235367, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 197, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.039888480, 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000202479 2023-12-11T19:38:00.121 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 68, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000013740, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000202 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 518256, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 69232 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 320, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 413, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 154481, 2023-12-11T19:38:00.122 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 370025, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.048782597, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.020366646, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.025912997, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.002515357, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 411, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.037971483, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000092388 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 614, 2023-12-11T19:38:00.123 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000188861, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000307 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 184944 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 53, 2023-12-11T19:38:00.124 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.002292463, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000043254 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000024498, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000012249 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 4, 2023-12-11T19:38:00.125 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 2, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000480478, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000240239 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 2, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 16, 2023-12-11T19:38:00.126 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 16, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 13, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 4, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 1, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 1, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 3, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 12, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 7, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 3, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2023-12-11T19:38:00.127 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 5, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 1073742070, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:38:00.128 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 2 2023-12-11T19:38:00.129 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 11, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 7, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 9, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2023-12-11T19:38:00.130 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 5, 2023-12-11T19:38:00.131 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 5, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 2, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195920, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2023-12-11T19:38:00.132 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.009216412, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002304103 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 20, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 22, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 19, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 22, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 3, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 16, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 20, 2023-12-11T19:38:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 4, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 1, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 2, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 1, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 49408, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 199036 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 4, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 4, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 9, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2023-12-11T19:38:00.134 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000480478, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000240239 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2023-12-11T19:38:00.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2023-12-11T19:38:00.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.137 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2023-12-11T19:38:00.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.139 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.142 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.143 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.144 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2023-12-11T19:38:00.145 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 1, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 1, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 261, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:38:00.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:38:00.147 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 255494, 2023-12-11T19:38:00.148 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 94, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7624, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 160, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-12-11T19:38:00.149 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 80037, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 327, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 41, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 12012, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 27, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 41, 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2023-12-11T19:38:00.150 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 41, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.319994440, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.007804742 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 41, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 64 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 41, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 0, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 41, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2023-12-11T19:38:00.151 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 4, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 7, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 6, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:38:00.152 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2023-12-11T19:38:00.153 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 88, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 8, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 8, 2023-12-11T19:38:00.154 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 16, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 1 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.155 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 604, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 705445, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:00.156 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 604, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 604, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 705445, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:00.157 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 41, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 12012, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 41, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 12012, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:00.158 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 41, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 41, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 41, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 41, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.159 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.160 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x5646b7c6a6e0": { 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 5, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1616, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 5, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1616, 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:00.161 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:00.162 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:00.162 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:00.162 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.162 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:00.162 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-12-11T19:38:00.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:00 smithi167 ceph-mon[102750]: pgmap v492: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-11T19:38:00.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:00 smithi167 ceph-mon[102750]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:38:00.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:59 smithi066 ceph-mon[97920]: pgmap v492: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-11T19:38:00.312 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:37:59 smithi066 ceph-mon[97920]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:38:00.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:00 smithi070 ceph-mon[104044]: pgmap v492: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-11T19:38:00.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:00 smithi070 ceph-mon[104044]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-11T19:38:00.582 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-12-11T19:38:00.582 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-12-11T19:38:00.582 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-12-11T19:38:00.583 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=11 2023-12-11T19:38:00.583 DEBUG:tasks.check_counter:Getting stats from i 2023-12-11T19:38:00.583 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.i perf dump 2023-12-11T19:38:01.361 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:38:02.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:02 smithi167 ceph-mon[102750]: pgmap v493: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:38:02.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:02 smithi066 ceph-mon[97920]: pgmap v493: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:38:02.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:02 smithi070 ceph-mon[104044]: pgmap v493: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 466, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 468, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 644894, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 264631, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.075788530, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.026850277, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.041751331, 2023-12-11T19:38:02.926 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.005613781, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 466, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.044740481, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000096009 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 584, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000152873, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000261 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 525520, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 68432 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.927 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 85, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 163, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 84358, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 272093, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 10, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.017648947, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.007151968, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.009940913, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000402072, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 163, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.037449890, 2023-12-11T19:38:02.928 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000229753 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 125, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000026174, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000209 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1088, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 187088 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 58, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 60, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 45462, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 129851, 2023-12-11T19:38:02.929 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 16, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 9, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.014080703, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.002983383, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.010444907, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000431884, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 58, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.032436556, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000559250 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 65, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000015475, 2023-12-11T19:38:02.930 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000238 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 38, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001860035, 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000048948 2023-12-11T19:38:02.931 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000033532, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000016766 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 2, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 2, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2023-12-11T19:38:02.932 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001738211, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000869105 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 16, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 16, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2023-12-11T19:38:02.933 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 13, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 4, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 2, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 2, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 3, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 10, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 7, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 3, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2023-12-11T19:38:02.934 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 5, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 1073742070, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:38:02.935 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.936 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 11, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 7, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 9, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:38:02.937 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 5, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2023-12-11T19:38:02.938 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 5, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 2, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195920, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4, 2023-12-11T19:38:02.939 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.008381039, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002095259 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 20, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 22, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 19, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 23, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 4, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 16, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 20, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 4, 2023-12-11T19:38:02.940 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 2, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 2, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 49404, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 199036 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 5, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 2, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 9, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 1, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.941 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001738211, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000869105 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.942 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.943 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.944 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.945 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2023-12-11T19:38:02.946 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2023-12-11T19:38:02.947 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2023-12-11T19:38:02.948 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2023-12-11T19:38:02.949 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.950 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:38:02.951 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2023-12-11T19:38:02.953 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.953 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.953 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 1, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 1, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 265, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.954 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:38:02.955 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:38:02.956 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 254871, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 91, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-12-11T19:38:02.957 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7624, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 160, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 80429, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 332, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.958 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 41, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 12073, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 27, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 41, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 41, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.267995320, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006536471 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 41, 2023-12-11T19:38:02.959 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 64 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 41, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 41, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 4, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 7, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2023-12-11T19:38:02.960 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 6, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2023-12-11T19:38:02.961 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2023-12-11T19:38:02.962 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 88, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 9, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 9, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 16, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 1 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2023-12-11T19:38:02.963 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2023-12-11T19:38:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 605, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 709456, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 605, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 605, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 709456, 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 41, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 12073, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 41, 2023-12-11T19:38:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 12073, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:02.967 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 41, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 41, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 41, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 41, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:02.968 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x560d9ae646e0": { 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 5, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1616, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 5, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1616, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:02.970 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-12-11T19:38:03.440 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-12-11T19:38:03.440 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-12-11T19:38:03.440 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2023-12-11T19:38:03.440 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2023-12-11T19:38:03.440 DEBUG:tasks.check_counter:Getting stats from l 2023-12-11T19:38:03.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 -- ceph daemon mds.l perf dump 2023-12-11T19:38:04.218 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T19:38:04.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:04 smithi167 ceph-mon[102750]: pgmap v494: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:04.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:04 smithi066 ceph-mon[97920]: pgmap v494: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:04.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:04 smithi070 ceph-mon[104044]: pgmap v494: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:05.883 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-12-11T19:38:05.883 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T19:38:05.883 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 372, 2023-12-11T19:38:05.883 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 374, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 99671, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 115756, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.043287849, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012547226, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.012046374, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.011631615, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 372, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.021099095, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000056717 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:05.884 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 302, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 386, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 76204, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 280228, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.040196103, 2023-12-11T19:38:05.885 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.013694691, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.010335020, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.011428804, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 385, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.022388648, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000058152 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3120, 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 187856 2023-12-11T19:38:05.886 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 495, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 487, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 614320, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 182498, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 3, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.075702654, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.026095825, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.028051276, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.017646158, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 486, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.036449923, 2023-12-11T19:38:05.887 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000074999 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 491344, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 68672 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2023-12-11T19:38:05.888 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1729, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.056356306, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000032594 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.889 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2023-12-11T19:38:05.890 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 1, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2023-12-11T19:38:05.891 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2023-12-11T19:38:05.892 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2023-12-11T19:38:05.893 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2023-12-11T19:38:05.894 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2023-12-11T19:38:05.895 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 5, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2023-12-11T19:38:05.896 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195920, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 4195422, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 5 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 3, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 3, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 0, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 1, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 1, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 0, 2023-12-11T19:38:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 47616, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207228 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2023-12-11T19:38:05.898 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.899 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.900 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.901 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.902 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.903 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.904 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.905 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.906 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.907 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.908 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1997, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-12-11T19:38:05.909 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T19:38:05.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-12-11T19:38:05.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 189239, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 48, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-12-11T19:38:05.912 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7624, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 160, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 8320, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 13, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-12-11T19:38:05.913 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1035, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1035, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1035, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.641988753, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000620278 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2023-12-11T19:38:05.914 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1035, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1036 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1035, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 1035, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 661, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 372, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2023-12-11T19:38:05.915 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T19:38:05.916 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2023-12-11T19:38:05.917 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 88, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 5, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 5, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.918 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2023-12-11T19:38:05.919 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 1165, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 675801, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 1165, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1165, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 675801, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1035, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1621, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1035, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1621, 2023-12-11T19:38:05.921 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1035, 2023-12-11T19:38:05.922 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1035, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1035, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1035, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:05.923 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x561cbd7c4a00": { 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-12-11T19:38:05.924 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-12-11T19:38:05.925 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.926 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-12-11T19:38:05.926 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-12-11T19:38:06.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:06 smithi167 ceph-mon[102750]: pgmap v495: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:06.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:06 smithi066 ceph-mon[97920]: pgmap v495: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:06.359 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:06 smithi070 ceph-mon[104044]: pgmap v495: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:06.407 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-12-11T19:38:06.407 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-12-11T19:38:06.407 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-12-11T19:38:06.407 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-12-11T19:38:06.408 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-12-11T19:38:06.419 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-12-11T19:38:06.420 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:38:06.420 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T19:38:06.473 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:38:06.473 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T19:38:06.522 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 -- 172.21.15.66:0/3090210101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c103030 msgr2=0x7f619c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 --2- 172.21.15.66:0/3090210101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c103030 0x7f619c1098e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0d0 tx=0x7f619400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 -- 172.21.15.66:0/3090210101 shutdown_connections 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 --2- 172.21.15.66:0/3090210101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c1014a0 0x7f619c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 --2- 172.21.15.66:0/3090210101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c103030 0x7f619c1098e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 --2- 172.21.15.66:0/3090210101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f619c102690 0x7f619c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:06.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 -- 172.21.15.66:0/3090210101 >> 172.21.15.66:0/3090210101 conn(0x7f619c0fca10 msgr2=0x7f619c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:06.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 -- 172.21.15.66:0/3090210101 shutdown_connections 2023-12-11T19:38:06.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.777+0000 7f61a47ff700 1 -- 172.21.15.66:0/3090210101 wait complete. 2023-12-11T19:38:06.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.778+0000 7f61a47ff700 1 Processor -- start 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.778+0000 7f61a47ff700 1 -- start start 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.778+0000 7f61a47ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c1014a0 0x7f619c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a47ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 0x7f619c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a47ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f619c103030 0x7f619c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a47ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19ef60 con 0x7f619c102690 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a47ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19f0a0 con 0x7f619c1014a0 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a47ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19f230 con 0x7f619c103030 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 0x7f619c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 0x7f619c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42626/0 (socket says 172.21.15.66:42626) 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 -- 172.21.15.66:0/4233997228 learned_addr learned my addr 172.21.15.66:0/4233997228 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a259b700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c1014a0 0x7f619c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a2d9c700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f619c103030 0x7f619c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:06.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 -- 172.21.15.66:0/4233997228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f619c103030 msgr2=0x7f619c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f619c103030 0x7f619c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 -- 172.21.15.66:0/4233997228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c1014a0 msgr2=0x7f619c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c1014a0 0x7f619c197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.779+0000 7f61a1d9a700 1 -- 172.21.15.66:0/4233997228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009d00 con 0x7f619c102690 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.780+0000 7f61a1d9a700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 0x7f619c197970 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9a0 tx=0x7f618c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.780+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c009790 con 0x7f619c102690 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.780+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c19f510 con 0x7f619c102690 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.780+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f618c00cbd0 con 0x7f619c102690 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.780+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c018680 con 0x7f619c102690 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.780+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c19fa70 con 0x7f619c102690 2023-12-11T19:38:06.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.781+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c108970 con 0x7f619c102690 2023-12-11T19:38:06.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.783+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f618c00cd40 con 0x7f619c102690 2023-12-11T19:38:06.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.784+0000 7f61937fe700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61880a1480 0x7f61880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:06.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.784+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f618c014070 con 0x7f619c102690 2023-12-11T19:38:06.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.785+0000 7f61a259b700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61880a1480 0x7f61880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:06.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.785+0000 7f61a259b700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61880a1480 0x7f61880a3940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f619c1024f0 tx=0x7f619800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:06.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:06.787+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f618c0967f0 con 0x7f619c102690 2023-12-11T19:38:07.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.001+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f619c068230 con 0x7f619c102690 2023-12-11T19:38:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.003+0000 7f61937fe700 1 -- 172.21.15.66:0/4233997228 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f618c026020 con 0x7f619c102690 2023-12-11T19:38:07.003 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:38:07.004 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:38:07.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.005+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61880a1480 msgr2=0x7f61880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.005+0000 7f61a47ff700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61880a1480 0x7f61880a3940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f619c1024f0 tx=0x7f619800b5b0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 msgr2=0x7f619c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 0x7f619c197970 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9a0 tx=0x7f618c00ed60 comp rx=0 tx=0).stop 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 shutdown_connections 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f61880a1480 0x7f61880a3940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c1014a0 0x7f619c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f619c103030 0x7f619c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 --2- 172.21.15.66:0/4233997228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f619c102690 0x7f619c197970 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 >> 172.21.15.66:0/4233997228 conn(0x7f619c0fca10 msgr2=0x7f619c105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 shutdown_connections 2023-12-11T19:38:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.006+0000 7f61a47ff700 1 -- 172.21.15.66:0/4233997228 wait complete. 2023-12-11T19:38:07.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:38:07.026 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:38:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 -- 172.21.15.66:0/1846003837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 msgr2=0x7f5f90106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 --2- 172.21.15.66:0/1846003837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90106d20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009990 tx=0x7f5f84009ca0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 -- 172.21.15.66:0/1846003837 shutdown_connections 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 --2- 172.21.15.66:0/1846003837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90102000 0x7f5f901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 --2- 172.21.15.66:0/1846003837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f90107260 0x7f5f901096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 --2- 172.21.15.66:0/1846003837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90106d20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 -- 172.21.15.66:0/1846003837 >> 172.21.15.66:0/1846003837 conn(0x7f5f900fb9f0 msgr2=0x7f5f900fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.237+0000 7f5f95fff700 1 -- 172.21.15.66:0/1846003837 shutdown_connections 2023-12-11T19:38:07.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.238+0000 7f5f95fff700 1 -- 172.21.15.66:0/1846003837 wait complete. 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.238+0000 7f5f95fff700 1 Processor -- start 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 -- start start 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f90102000 0x7f5f90197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90107260 0x7f5f9019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9019ef40 con 0x7f5f90104930 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9019f080 con 0x7f5f90107260 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f95fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9019f210 con 0x7f5f90102000 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f8effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f8effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42642/0 (socket says 172.21.15.66:42642) 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f8effd700 1 -- 172.21.15.66:0/3116381229 learned_addr learned my addr 172.21.15.66:0/3116381229 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.239+0000 7f5f8f7fe700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f90102000 0x7f5f90197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8ffff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90107260 0x7f5f9019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8effd700 1 -- 172.21.15.66:0/3116381229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f90102000 msgr2=0x7f5f90197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8effd700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f90102000 0x7f5f90197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8effd700 1 -- 172.21.15.66:0/3116381229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90107260 msgr2=0x7f5f9019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8effd700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90107260 0x7f5f9019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8effd700 1 -- 172.21.15.66:0/3116381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f840095f0 con 0x7f5f90104930 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8ffff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90107260 0x7f5f9019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8effd700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90197950 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400b2e0 tx=0x7f5f84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8401d070 con 0x7f5f90104930 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f8400beb0 con 0x7f5f90104930 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84021410 con 0x7f5f90104930 2023-12-11T19:38:07.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9019f490 con 0x7f5f90104930 2023-12-11T19:38:07.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.240+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9019f860 con 0x7f5f90104930 2023-12-11T19:38:07.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.242+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f900fd570 con 0x7f5f90104930 2023-12-11T19:38:07.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.243+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5f8400f640 con 0x7f5f90104930 2023-12-11T19:38:07.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.245+0000 7f5f8cff9700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5f780a1480 0x7f5f780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.245+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5f840ce100 con 0x7f5f90104930 2023-12-11T19:38:07.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.245+0000 7f5f8f7fe700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5f780a1480 0x7f5f780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.246+0000 7f5f8f7fe700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5f780a1480 0x7f5f780a3940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f8000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:07.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.249+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5f840977b0 con 0x7f5f90104930 2023-12-11T19:38:07.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.467+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f900fcf50 con 0x7f5f90104930 2023-12-11T19:38:07.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.467+0000 7f5f8cff9700 1 -- 172.21.15.66:0/3116381229 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f8409add0 con 0x7f5f90104930 2023-12-11T19:38:07.467 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:38:07.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5f780a1480 msgr2=0x7f5f780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5f780a1480 0x7f5f780a3940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f8000b410 comp rx=0 tx=0).stop 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 msgr2=0x7f5f90197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90197950 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400b2e0 tx=0x7f5f84004730 comp rx=0 tx=0).stop 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 shutdown_connections 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f5f780a1480 0x7f5f780a3940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f90107260 0x7f5f9019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f90102000 0x7f5f90197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 --2- 172.21.15.66:0/3116381229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f90104930 0x7f5f90197950 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 >> 172.21.15.66:0/3116381229 conn(0x7f5f900fb9f0 msgr2=0x7f5f90103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 shutdown_connections 2023-12-11T19:38:07.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.470+0000 7f5f95fff700 1 -- 172.21.15.66:0/3116381229 wait complete. 2023-12-11T19:38:07.489 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 -- 172.21.15.66:0/350754164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa101015a0 msgr2=0x7ffa101019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 --2- 172.21.15.66:0/350754164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa101015a0 0x7ffa101019c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0099c0 tx=0x7ffa0c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 -- 172.21.15.66:0/350754164 shutdown_connections 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 --2- 172.21.15.66:0/350754164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa10103130 0x7ffa101099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 --2- 172.21.15.66:0/350754164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa101015a0 0x7ffa101019c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 --2- 172.21.15.66:0/350754164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa10102790 0x7ffa10102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 -- 172.21.15.66:0/350754164 >> 172.21.15.66:0/350754164 conn(0x7ffa100fcb10 msgr2=0x7ffa100fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:07.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 -- 172.21.15.66:0/350754164 shutdown_connections 2023-12-11T19:38:07.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.701+0000 7ffa175ff700 1 -- 172.21.15.66:0/350754164 wait complete. 2023-12-11T19:38:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.702+0000 7ffa175ff700 1 Processor -- start 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.702+0000 7ffa175ff700 1 -- start start 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa175ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa101015a0 0x7ffa10199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa175ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa10102790 0x7ffa10199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa175ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 0x7ffa101a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa175ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101a11a0 con 0x7ffa10103130 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa175ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101a12e0 con 0x7ffa10102790 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa175ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101a1470 con 0x7ffa101015a0 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 0x7ffa101a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 0x7ffa101a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42654/0 (socket says 172.21.15.66:42654) 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 -- 172.21.15.66:0/1120975770 learned_addr learned my addr 172.21.15.66:0/1120975770 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa1539b700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa101015a0 0x7ffa10199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa14b9a700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa10102790 0x7ffa10199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 -- 172.21.15.66:0/1120975770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa101015a0 msgr2=0x7ffa10199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa101015a0 0x7ffa10199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 -- 172.21.15.66:0/1120975770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa10102790 msgr2=0x7ffa10199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.703+0000 7ffa15b9c700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa10102790 0x7ffa10199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa15b9c700 1 -- 172.21.15.66:0/1120975770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c0095f0 con 0x7ffa10103130 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa15b9c700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 0x7ffa101a0bd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ffa08007d70 tx=0x7ffa0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800ac90 con 0x7ffa10103130 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa0800adf0 con 0x7ffa10103130 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa101a1750 con 0x7ffa10103130 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080115e0 con 0x7ffa10103130 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.704+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa101a1c10 con 0x7ffa10103130 2023-12-11T19:38:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.705+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa10068310 con 0x7ffa10103130 2023-12-11T19:38:07.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.707+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ffa08012420 con 0x7ffa10103130 2023-12-11T19:38:07.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.709+0000 7ffa067fc700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9fc0a1480 0x7ff9fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:07.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.709+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffa080ccf50 con 0x7ffa10103130 2023-12-11T19:38:07.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.709+0000 7ffa1539b700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9fc0a1480 0x7ff9fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:07.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.710+0000 7ffa1539b700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9fc0a1480 0x7ff9fc0a3940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c006010 tx=0x7ffa0c003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:07.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.713+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ffa08096570 con 0x7ffa10103130 2023-12-11T19:38:07.811 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:07 smithi066 ceph-mon[97920]: pgmap v496: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-11T19:38:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:07 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4233997228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:38:07.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:07 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3116381229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:38:07.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:07 smithi070 ceph-mon[104044]: pgmap v496: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-11T19:38:07.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:07 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4233997228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:38:07.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:07 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3116381229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:38:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.925+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa101a1e90 con 0x7ffa10103130 2023-12-11T19:38:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.926+0000 7ffa067fc700 1 -- 172.21.15.66:0/1120975770 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa08099b90 con 0x7ffa10103130 2023-12-11T19:38:07.926 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:38:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.928+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9fc0a1480 msgr2=0x7ff9fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.928+0000 7ffa175ff700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9fc0a1480 0x7ff9fc0a3940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c006010 tx=0x7ffa0c003740 comp rx=0 tx=0).stop 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.928+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 msgr2=0x7ffa101a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.928+0000 7ffa175ff700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 0x7ffa101a0bd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ffa08007d70 tx=0x7ffa0800a340 comp rx=0 tx=0).stop 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 shutdown_connections 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7ff9fc0a1480 0x7ff9fc0a3940 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa10103130 0x7ffa101a0bd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa101015a0 0x7ffa10199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 --2- 172.21.15.66:0/1120975770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa10102790 0x7ffa10199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 >> 172.21.15.66:0/1120975770 conn(0x7ffa100fcb10 msgr2=0x7ffa10107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 shutdown_connections 2023-12-11T19:38:07.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:07.929+0000 7ffa175ff700 1 -- 172.21.15.66:0/1120975770 wait complete. 2023-12-11T19:38:07.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:38:08.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:07 smithi167 ceph-mon[102750]: pgmap v496: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-11T19:38:08.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:07 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4233997228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:38:08.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:07 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3116381229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:38:08.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 -- 172.21.15.66:0/746528837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174102000 msgr2=0x7fb1741043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/746528837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174102000 0x7fb1741043f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb170009990 tx=0x7fb170009ca0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 -- 172.21.15.66:0/746528837 shutdown_connections 2023-12-11T19:38:08.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/746528837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb174107260 0x7fb1741096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/746528837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174102000 0x7fb1741043f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/746528837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb174104930 0x7fb174106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 -- 172.21.15.66:0/746528837 >> 172.21.15.66:0/746528837 conn(0x7fb1740fb9f0 msgr2=0x7fb1740fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:08.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 -- 172.21.15.66:0/746528837 shutdown_connections 2023-12-11T19:38:08.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.157+0000 7fb17b5ff700 1 -- 172.21.15.66:0/746528837 wait complete. 2023-12-11T19:38:08.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.158+0000 7fb17b5ff700 1 Processor -- start 2023-12-11T19:38:08.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 -- start start 2023-12-11T19:38:08.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb174102000 0x7fb174199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb174104930 0x7fb174199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 0x7fb1741a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741a1150 con 0x7fb174107260 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741a1290 con 0x7fb174102000 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17b5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741a1420 con 0x7fb174104930 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb179b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 0x7fb1741a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb179b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 0x7fb1741a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42674/0 (socket says 172.21.15.66:42674) 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb179b9c700 1 -- 172.21.15.66:0/4076048089 learned_addr learned my addr 172.21.15.66:0/4076048089 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb17939b700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb174102000 0x7fb174199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.159+0000 7fb178b9a700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb174104930 0x7fb174199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb179b9c700 1 -- 172.21.15.66:0/4076048089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb174104930 msgr2=0x7fb174199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb179b9c700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb174104930 0x7fb174199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb179b9c700 1 -- 172.21.15.66:0/4076048089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb174102000 msgr2=0x7fb174199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb179b9c700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb174102000 0x7fb174199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb179b9c700 1 -- 172.21.15.66:0/4076048089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1700095f0 con 0x7fb174107260 2023-12-11T19:38:08.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb179b9c700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 0x7fb1741a0b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb16c007ee0 tx=0x7fb16c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:08.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c017070 con 0x7fb174107260 2023-12-11T19:38:08.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb16c00f040 con 0x7fb174107260 2023-12-11T19:38:08.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1741a1700 con 0x7fb174107260 2023-12-11T19:38:08.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c005710 con 0x7fb174107260 2023-12-11T19:38:08.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.160+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1741a1c10 con 0x7fb174107260 2023-12-11T19:38:08.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.161+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb174068310 con 0x7fb174107260 2023-12-11T19:38:08.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.163+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb16c005870 con 0x7fb174107260 2023-12-11T19:38:08.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.165+0000 7fb16a7fc700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1600a1500 0x7fb1600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.165+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb16c0ccb50 con 0x7fb174107260 2023-12-11T19:38:08.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.165+0000 7fb17939b700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1600a1500 0x7fb1600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.165+0000 7fb17939b700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1600a1500 0x7fb1600a39c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb170006010 tx=0x7fb17000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:08.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.167+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb16c096170 con 0x7fb174107260 2023-12-11T19:38:08.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.382+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1741a1ed0 con 0x7fb174107260 2023-12-11T19:38:08.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.383+0000 7fb16a7fc700 1 -- 172.21.15.66:0/4076048089 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb16c099790 con 0x7fb174107260 2023-12-11T19:38:08.383 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1600a1500 msgr2=0x7fb1600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1600a1500 0x7fb1600a39c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb170006010 tx=0x7fb17000b610 comp rx=0 tx=0).stop 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 msgr2=0x7fb1741a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 0x7fb1741a0b80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb16c007ee0 tx=0x7fb16c00d3b0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 shutdown_connections 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7fb1600a1500 0x7fb1600a39c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb174102000 0x7fb174199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb174107260 0x7fb1741a0b80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 --2- 172.21.15.66:0/4076048089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb174104930 0x7fb174199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 >> 172.21.15.66:0/4076048089 conn(0x7fb1740fb9f0 msgr2=0x7fb1741000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:08.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 shutdown_connections 2023-12-11T19:38:08.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.385+0000 7fb17b5ff700 1 -- 172.21.15.66:0/4076048089 wait complete. 2023-12-11T19:38:08.404 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 -- 172.21.15.66:0/4265876938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c074030 msgr2=0x7f274c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 --2- 172.21.15.66:0/4265876938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c074030 0x7f274c0744b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f273c009990 tx=0x7f273c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 -- 172.21.15.66:0/4265876938 shutdown_connections 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 --2- 172.21.15.66:0/4265876938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c109460 0x7f274c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 --2- 172.21.15.66:0/4265876938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c075610 0x7f274c073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 --2- 172.21.15.66:0/4265876938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c074030 0x7f274c0744b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 -- 172.21.15.66:0/4265876938 >> 172.21.15.66:0/4265876938 conn(0x7f274c0fcb10 msgr2=0x7f274c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 -- 172.21.15.66:0/4265876938 shutdown_connections 2023-12-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.614+0000 7f27533ff700 1 -- 172.21.15.66:0/4265876938 wait complete. 2023-12-11T19:38:08.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 Processor -- start 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 -- start start 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c074030 0x7f274c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c075610 0x7f274c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 0x7f274c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c1a11b0 con 0x7f274c109460 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c1a12f0 con 0x7f274c075610 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.615+0000 7f27533ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c1a1480 con 0x7f274c074030 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 0x7f274c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 0x7f274c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42688/0 (socket says 172.21.15.66:42688) 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 -- 172.21.15.66:0/874100839 learned_addr learned my addr 172.21.15.66:0/874100839 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275119b700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c074030 0x7f274c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275099a700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c075610 0x7f274c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 -- 172.21.15.66:0/874100839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c074030 msgr2=0x7f274c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c074030 0x7f274c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 -- 172.21.15.66:0/874100839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c075610 msgr2=0x7f274c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c075610 0x7f274c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 -- 172.21.15.66:0/874100839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0095f0 con 0x7f274c109460 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275099a700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c075610 0x7f274c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.616+0000 7f275199c700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 0x7f274c1a0be0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2744007ee0 tx=0x7f274400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.617+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744017070 con 0x7f274c109460 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.617+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f274400f040 con 0x7f274c109460 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.617+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c1a1760 con 0x7f274c109460 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.617+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744005710 con 0x7f274c109460 2023-12-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.617+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c1a1c20 con 0x7f274c109460 2023-12-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.618+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c108220 con 0x7f274c109460 2023-12-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.620+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2744005870 con 0x7f274c109460 2023-12-11T19:38:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.621+0000 7f27427fc700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27380a1480 0x7f27380a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.621+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f27440cc4d0 con 0x7f274c109460 2023-12-11T19:38:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.622+0000 7f275119b700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27380a1480 0x7f27380a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:08.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.622+0000 7f275119b700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27380a1480 0x7f27380a3940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2748007c60 tx=0x7f27480073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:08.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.625+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2744095af0 con 0x7f274c109460 2023-12-11T19:38:08.811 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:08 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1120975770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:38:08.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:08 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/4076048089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:38:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.853+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f274c10ee40 con 0x7f274c109460 2023-12-11T19:38:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.853+0000 7f27427fc700 1 -- 172.21.15.66:0/874100839 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+590 (secure 0 0 0) 0x7f2744099110 con 0x7f274c109460 2023-12-11T19:38:08.853 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2642090748 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:08.853 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/4268281106 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:08.853 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/153702681 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1199617033 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1568045414 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/294726661 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3525772822 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3525772822 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1790110256 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:08.854 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/4268281106 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27380a1480 msgr2=0x7f27380a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27380a1480 0x7f27380a3940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2748007c60 tx=0x7f27480073d0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 msgr2=0x7f274c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 0x7f274c1a0be0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2744007ee0 tx=0x7f274400d3b0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 shutdown_connections 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f27380a1480 0x7f27380a3940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f274c109460 0x7f274c1a0be0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c074030 0x7f274c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 --2- 172.21.15.66:0/874100839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f274c075610 0x7f274c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 >> 172.21.15.66:0/874100839 conn(0x7f274c0fcb10 msgr2=0x7f274c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 shutdown_connections 2023-12-11T19:38:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:08.856+0000 7f27533ff700 1 -- 172.21.15.66:0/874100839 wait complete. 2023-12-11T19:38:08.857 INFO:teuthology.orchestra.run.smithi066.stderr:listed 10 entries 2023-12-11T19:38:08.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:08 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1120975770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:38:08.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:08 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/4076048089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:38:08.875 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-12-11T19:38:08.875 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T19:38:08.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T19:38:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:08 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1120975770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:38:09.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:08 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/4076048089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 -- 172.21.15.66:0/2413356010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268102690 msgr2=0x7f4268102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 --2- 172.21.15.66:0/2413356010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268102690 0x7f4268102af0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f425c0099c0 tx=0x7f425c009cd0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 -- 172.21.15.66:0/2413356010 shutdown_connections 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 --2- 172.21.15.66:0/2413356010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4268103030 0x7f42681098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 --2- 172.21.15.66:0/2413356010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42681014a0 0x7f42681018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 --2- 172.21.15.66:0/2413356010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268102690 0x7f4268102af0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 -- 172.21.15.66:0/2413356010 >> 172.21.15.66:0/2413356010 conn(0x7f42680fca10 msgr2=0x7f42680fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 -- 172.21.15.66:0/2413356010 shutdown_connections 2023-12-11T19:38:09.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.172+0000 7f426e3ff700 1 -- 172.21.15.66:0/2413356010 wait complete. 2023-12-11T19:38:09.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.173+0000 7f426e3ff700 1 Processor -- start 2023-12-11T19:38:09.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 -- start start 2023-12-11T19:38:09.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42681014a0 0x7f4268197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4268102690 0x7f4268197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 0x7f426819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426819ee70 con 0x7f4268103030 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426819efb0 con 0x7f4268102690 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426e3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426819f140 con 0x7f42681014a0 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 0x7f426819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 0x7f426819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42712/0 (socket says 172.21.15.66:42712) 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f426c99c700 1 -- 172.21.15.66:0/1994787516 learned_addr learned my addr 172.21.15.66:0/1994787516 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f42677fe700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4268102690 0x7f4268197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.174+0000 7f4267fff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42681014a0 0x7f4268197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426c99c700 1 -- 172.21.15.66:0/1994787516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42681014a0 msgr2=0x7f4268197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426c99c700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42681014a0 0x7f4268197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426c99c700 1 -- 172.21.15.66:0/1994787516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4268102690 msgr2=0x7f4268197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426c99c700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4268102690 0x7f4268197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426c99c700 1 -- 172.21.15.66:0/1994787516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0095f0 con 0x7f4268103030 2023-12-11T19:38:09.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426c99c700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 0x7f426819e8a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4260007ee0 tx=0x7f426000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:09.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260017070 con 0x7f4268103030 2023-12-11T19:38:09.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f426000f040 con 0x7f4268103030 2023-12-11T19:38:09.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260005570 con 0x7f4268103030 2023-12-11T19:38:09.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426819f420 con 0x7f4268103030 2023-12-11T19:38:09.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.175+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426819f8e0 con 0x7f4268103030 2023-12-11T19:38:09.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.177+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268108940 con 0x7f4268103030 2023-12-11T19:38:09.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.179+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f426000db20 con 0x7f4268103030 2023-12-11T19:38:09.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.180+0000 7f42657fa700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42500a1500 0x7f42500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.180+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42600ccab0 con 0x7f4268103030 2023-12-11T19:38:09.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.181+0000 7f4267fff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42500a1500 0x7f42500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.181+0000 7f4267fff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42500a1500 0x7f42500a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f42681024f0 tx=0x7f425800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:09.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.183+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f42600960d0 con 0x7f4268103030 2023-12-11T19:38:09.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.405+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4268068230 con 0x7f4268103030 2023-12-11T19:38:09.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.407+0000 7f42657fa700 1 -- 172.21.15.66:0/1994787516 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22488 (secure 0 0 0) 0x7f42600996f0 con 0x7f4268103030 2023-12-11T19:38:09.407 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-12-11T19:38:09.407 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":88,"fsid":"976cf7f8-985a-11ee-95a3-87774f69a715","created":"2023-12-11T19:23:09.144370+0000","modified":"2023-12-11T19:32:11.575017+0000","last_up_change":"2023-12-11T19:30:57.323165+0000","last_in_change":"2023-12-11T19:30:36.365525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T19:26:55.011806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T19:31:49.794469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T19:31:51.228031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5da1c6d-b901-47aa-b2e3-4e377f852890","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6803","nonce":345409379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6805","nonce":345409379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6809","nonce":345409379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":345409379},{"type":"v1","addr":"172.21.15.66:6807","nonce":345409379}]},"public_addr":"172.21.15.66:6803/345409379","cluster_addr":"172.21.15.66:6805/345409379","heartbeat_back_addr":"172.21.15.66:6809/345409379","heartbeat_front_addr":"172.21.15.66:6807/345409379","state":["exists","up"]},{"osd":1,"uuid":"820e245f-dede-4543-a91d-d7ba2568ff7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6801","nonce":1682554189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6803","nonce":1682554189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6807","nonce":1682554189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1682554189},{"type":"v1","addr":"172.21.15.70:6805","nonce":1682554189}]},"public_addr":"172.21.15.70:6801/1682554189","cluster_addr":"172.21.15.70:6803/1682554189","heartbeat_back_addr":"172.21.15.70:6807/1682554189","heartbeat_front_addr":"172.21.15.70:6805/1682554189","state":["exists","up"]},{"osd":2,"uuid":"0cb5fd9c-e264-4947-85e5-413b230c4575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6801","nonce":440232508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6803","nonce":440232508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6807","nonce":440232508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":440232508},{"type":"v1","addr":"172.21.15.167:6805","nonce":440232508}]},"public_addr":"172.21.15.167:6801/440232508","cluster_addr":"172.21.15.167:6803/440232508","heartbeat_back_addr":"172.21.15.167:6807/440232508","heartbeat_front_addr":"172.21.15.167:6805/440232508","state":["exists","up"]},{"osd":3,"uuid":"c2c58a66-528c-490e-8f4d-6fdb5be75536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6811","nonce":1707492505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6813","nonce":1707492505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6817","nonce":1707492505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1707492505},{"type":"v1","addr":"172.21.15.66:6815","nonce":1707492505}]},"public_addr":"172.21.15.66:6811/1707492505","cluster_addr":"172.21.15.66:6813/1707492505","heartbeat_back_addr":"172.21.15.66:6817/1707492505","heartbeat_front_addr":"172.21.15.66:6815/1707492505","state":["exists","up"]},{"osd":4,"uuid":"181213ac-13a3-4539-9435-df0f7299186b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6809","nonce":161576683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6811","nonce":161576683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6815","nonce":161576683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":161576683},{"type":"v1","addr":"172.21.15.70:6813","nonce":161576683}]},"public_addr":"172.21.15.70:6809/161576683","cluster_addr":"172.21.15.70:6811/161576683","heartbeat_back_addr":"172.21.15.70:6815/161576683","heartbeat_front_addr":"172.21.15.70:6813/161576683","state":["exists","up"]},{"osd":5,"uuid":"df3fbfc0-cd26-4ac1-906c-28bd60f02c38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6809","nonce":1023707142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6811","nonce":1023707142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6815","nonce":1023707142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1023707142},{"type":"v1","addr":"172.21.15.167:6813","nonce":1023707142}]},"public_addr":"172.21.15.167:6809/1023707142","cluster_addr":"172.21.15.167:6811/1023707142","heartbeat_back_addr":"172.21.15.167:6815/1023707142","heartbeat_front_addr":"172.21.15.167:6813/1023707142","state":["exists","up"]},{"osd":6,"uuid":"bb558c6d-a82c-4255-a040-4432a98c2aa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6819","nonce":7519239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6821","nonce":7519239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6825","nonce":7519239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":7519239},{"type":"v1","addr":"172.21.15.66:6823","nonce":7519239}]},"public_addr":"172.21.15.66:6819/7519239","cluster_addr":"172.21.15.66:6821/7519239","heartbeat_back_addr":"172.21.15.66:6825/7519239","heartbeat_front_addr":"172.21.15.66:6823/7519239","state":["exists","up"]},{"osd":7,"uuid":"ff1ec2fc-e59d-4255-8f19-2f841bb4dc8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6817","nonce":4038002078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6819","nonce":4038002078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6823","nonce":4038002078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":4038002078},{"type":"v1","addr":"172.21.15.70:6821","nonce":4038002078}]},"public_addr":"172.21.15.70:6817/4038002078","cluster_addr":"172.21.15.70:6819/4038002078","heartbeat_back_addr":"172.21.15.70:6823/4038002078","heartbeat_front_addr":"172.21.15.70:6821/4038002078","state":["exists","up"]},{"osd":8,"uuid":"5d35b137-1e21-4e15-b0b8-4f0c6c693e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6817","nonce":2623463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6819","nonce":2623463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6823","nonce":2623463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2623463019},{"type":"v1","addr":"172.21.15.167:6821","nonce":2623463019}]},"public_addr":"172.21.15.167:6817/2623463019","cluster_addr":"172.21.15.167:6819/2623463019","heartbeat_back_addr":"172.21.15.167:6823/2623463019","heartbeat_front_addr":"172.21.15.167:6821/2623463019","state":["exists","up"]},{"osd":9,"uuid":"241627a2-87be-4277-9306-48d9f64595a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6827","nonce":969904470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6829","nonce":969904470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6833","nonce":969904470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":969904470},{"type":"v1","addr":"172.21.15.66:6831","nonce":969904470}]},"public_addr":"172.21.15.66:6827/969904470","cluster_addr":"172.21.15.66:6829/969904470","heartbeat_back_addr":"172.21.15.66:6833/969904470","heartbeat_front_addr":"172.21.15.66:6831/969904470","state":["exists","up"]},{"osd":10,"uuid":"afbe1b4e-b374-46a0-884f-89c5a4cf84ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6825","nonce":2336173720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6827","nonce":2336173720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6831","nonce":2336173720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2336173720},{"type":"v1","addr":"172.21.15.70:6829","nonce":2336173720}]},"public_addr":"172.21.15.70:6825/2336173720","cluster_addr":"172.21.15.70:6827/2336173720","heartbeat_back_addr":"172.21.15.70:6831/2336173720","heartbeat_front_addr":"172.21.15.70:6829/2336173720","state":["exists","up"]},{"osd":11,"uuid":"af4e5d83-929b-49e5-af0e-9e2524e9d264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6825","nonce":819959610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6827","nonce":819959610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6831","nonce":819959610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":819959610},{"type":"v1","addr":"172.21.15.167:6829","nonce":819959610}]},"public_addr":"172.21.15.167:6825/819959610","cluster_addr":"172.21.15.167:6827/819959610","heartbeat_back_addr":"172.21.15.167:6831/819959610","heartbeat_front_addr":"172.21.15.167:6829/819959610","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:25:55.490551+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:23.078022+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:26:50.571991+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:18.122534+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:27:44.980705+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:11.423659+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:28:38.784421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:06.162552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:29:33.539806+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:00.818938+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:28.112078+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T19:30:55.429683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/2642090748":"2023-12-12T19:24:10.190965+0000","172.21.15.66:6801/4268281106":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/153702681":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/1199617033":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1568045414":"2023-12-12T19:24:10.190965+0000","172.21.15.66:0/294726661":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6801/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/3525772822":"2023-12-12T19:23:41.756564+0000","172.21.15.66:0/1790110256":"2023-12-12T19:23:41.756564+0000","172.21.15.66:6800/4268281106":"2023-12-12T19:24:10.190965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T19:38:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42500a1500 msgr2=0x7f42500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42500a1500 0x7f42500a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f42681024f0 tx=0x7f425800b410 comp rx=0 tx=0).stop 2023-12-11T19:38:09.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 msgr2=0x7f426819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 0x7f426819e8a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4260007ee0 tx=0x7f426000d3b0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 shutdown_connections 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f42500a1500 0x7f42500a39c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4268103030 0x7f426819e8a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42681014a0 0x7f4268197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 --2- 172.21.15.66:0/1994787516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4268102690 0x7f4268197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 >> 172.21.15.66:0/1994787516 conn(0x7f42680fca10 msgr2=0x7f4268105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 shutdown_connections 2023-12-11T19:38:09.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.410+0000 7f426e3ff700 1 -- 172.21.15.66:0/1994787516 wait complete. 2023-12-11T19:38:09.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T19:26:55.011806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T19:31:49.794469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T19:31:51.228031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T19:38:09.430 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T19:38:09.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1266558898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28104930 msgr2=0x7f6a28106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1266558898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28104930 0x7f6a28106d20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009990 tx=0x7f6a1c009ca0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1266558898 shutdown_connections 2023-12-11T19:38:09.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1266558898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28107260 0x7f6a281096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1266558898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a28102000 0x7f6a281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1266558898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28104930 0x7f6a28106d20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1266558898 >> 172.21.15.66:0/1266558898 conn(0x7f6a280fb9f0 msgr2=0x7f6a280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:09.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1266558898 shutdown_connections 2023-12-11T19:38:09.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.642+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1266558898 wait complete. 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.643+0000 7f6a2dbff700 1 Processor -- start 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.643+0000 7f6a2dbff700 1 -- start start 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a2dbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a28102000 0x7f6a28199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a2dbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28104930 0x7f6a28199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a2dbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 0x7f6a281a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a2dbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a281a1180 con 0x7f6a28107260 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a2dbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a281a12c0 con 0x7f6a28104930 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a2dbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a281a1450 con 0x7f6a28102000 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 0x7f6a281a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 0x7f6a281a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42736/0 (socket says 172.21.15.66:42736) 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a27fff700 1 -- 172.21.15.66:0/1392686315 learned_addr learned my addr 172.21.15.66:0/1392686315 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:09.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a277fe700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a28102000 0x7f6a28199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.644+0000 7f6a26ffd700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28104930 0x7f6a28199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a27fff700 1 -- 172.21.15.66:0/1392686315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a28102000 msgr2=0x7f6a28199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a27fff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a28102000 0x7f6a28199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a27fff700 1 -- 172.21.15.66:0/1392686315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28104930 msgr2=0x7f6a28199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a27fff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28104930 0x7f6a28199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a27fff700 1 -- 172.21.15.66:0/1392686315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0095f0 con 0x7f6a28107260 2023-12-11T19:38:09.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a27fff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 0x7f6a281a0bb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000ef30 tx=0x7f6a2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20018070 con 0x7f6a28107260 2023-12-11T19:38:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a2000f040 con 0x7f6a28107260 2023-12-11T19:38:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a281a16d0 con 0x7f6a28107260 2023-12-11T19:38:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000cbc0 con 0x7f6a28107260 2023-12-11T19:38:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.645+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a281a1b90 con 0x7f6a28107260 2023-12-11T19:38:09.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.646+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a28108e60 con 0x7f6a28107260 2023-12-11T19:38:09.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.648+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6a2000cd80 con 0x7f6a28107260 2023-12-11T19:38:09.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.650+0000 7f6a24ff9700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6a100a1480 0x7f6a100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:09.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.650+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a200cc820 con 0x7f6a28107260 2023-12-11T19:38:09.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.650+0000 7f6a277fe700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6a100a1480 0x7f6a100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:09.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.651+0000 7f6a277fe700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6a100a1480 0x7f6a100a3940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005950 tx=0x7f6a1800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:09.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.654+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6a20095ed0 con 0x7f6a28107260 2023-12-11T19:38:09.811 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:09 smithi066 ceph-mon[97920]: pgmap v497: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-11T19:38:09.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:09 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/874100839' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T19:38:09.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:09 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1994787516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:38:09.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:09 smithi070 ceph-mon[104044]: pgmap v497: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-11T19:38:09.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/874100839' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T19:38:09.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:09 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1994787516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:38:09.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.867+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a2810cee0 con 0x7f6a28107260 2023-12-11T19:38:09.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.867+0000 7f6a24ff9700 1 -- 172.21.15.66:0/1392686315 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a200994f0 con 0x7f6a28107260 2023-12-11T19:38:09.867 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-12-11T19:38:09.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6a100a1480 msgr2=0x7f6a100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6a100a1480 0x7f6a100a3940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005950 tx=0x7f6a1800b410 comp rx=0 tx=0).stop 2023-12-11T19:38:09.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 msgr2=0x7f6a281a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:09.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 0x7f6a281a0bb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000ef30 tx=0x7f6a2000c3a0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 shutdown_connections 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f6a100a1480 0x7f6a100a3940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a28107260 0x7f6a281a0bb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a28102000 0x7f6a28199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 --2- 172.21.15.66:0/1392686315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28104930 0x7f6a28199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 >> 172.21.15.66:0/1392686315 conn(0x7f6a280fb9f0 msgr2=0x7f6a281015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.869+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 shutdown_connections 2023-12-11T19:38:09.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:09.870+0000 7f6a2dbff700 1 -- 172.21.15.66:0/1392686315 wait complete. 2023-12-11T19:38:09.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T19:38:10.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:09 smithi167 ceph-mon[102750]: pgmap v497: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-11T19:38:10.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:09 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/874100839' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T19:38:10.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:09 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1994787516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.099+0000 7f2d237ff700 1 -- 172.21.15.66:0/3513095944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c103130 msgr2=0x7f2d1c1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.099+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3513095944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c103130 0x7f2d1c1099e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400b0d0 tx=0x7f2d1400b3e0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 -- 172.21.15.66:0/3513095944 shutdown_connections 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3513095944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d1c1015a0 0x7f2d1c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3513095944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c103130 0x7f2d1c1099e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3513095944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c102790 0x7f2d1c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 -- 172.21.15.66:0/3513095944 >> 172.21.15.66:0/3513095944 conn(0x7f2d1c0fcb10 msgr2=0x7f2d1c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 -- 172.21.15.66:0/3513095944 shutdown_connections 2023-12-11T19:38:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.100+0000 7f2d237ff700 1 -- 172.21.15.66:0/3513095944 wait complete. 2023-12-11T19:38:10.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.101+0000 7f2d237ff700 1 Processor -- start 2023-12-11T19:38:10.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.101+0000 7f2d237ff700 1 -- start start 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.101+0000 7f2d237ff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d1c1015a0 0x7f2d1c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.101+0000 7f2d237ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 0x7f2d1c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.101+0000 7f2d237ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c103130 0x7f2d1c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d237ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0746a0 con 0x7f2d1c102790 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d237ff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c074810 con 0x7f2d1c1015a0 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d237ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c1a1590 con 0x7f2d1c103130 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 0x7f2d1c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 0x7f2d1c0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42762/0 (socket says 172.21.15.66:42762) 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 -- 172.21.15.66:0/3009415814 learned_addr learned my addr 172.21.15.66:0/3009415814 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d21d9c700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c103130 0x7f2d1c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d2159b700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d1c1015a0 0x7f2d1c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 -- 172.21.15.66:0/3009415814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c103130 msgr2=0x7f2d1c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c103130 0x7f2d1c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 -- 172.21.15.66:0/3009415814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d1c1015a0 msgr2=0x7f2d1c075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d1c1015a0 0x7f2d1c075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 -- 172.21.15.66:0/3009415814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009520 con 0x7f2d1c102790 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d21d9c700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c103130 0x7f2d1c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.102+0000 7f2d20d9a700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 0x7f2d1c0739c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9a0 tx=0x7f2d0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.103+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c009790 con 0x7f2d1c102790 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.103+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d0c010430 con 0x7f2d1c102790 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.103+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c018680 con 0x7f2d1c102790 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.103+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d14009d00 con 0x7f2d1c102790 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.103+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c1a1b50 con 0x7f2d1c102790 2023-12-11T19:38:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.104+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1c108a20 con 0x7f2d1c102790 2023-12-11T19:38:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.106+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2d0c0105a0 con 0x7f2d1c102790 2023-12-11T19:38:10.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.107+0000 7f2d127fc700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2d080a1500 0x7f2d080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.107+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2d0c014070 con 0x7f2d1c102790 2023-12-11T19:38:10.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.108+0000 7f2d2159b700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2d080a1500 0x7f2d080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.108+0000 7f2d2159b700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2d080a1500 0x7f2d080a39c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1025f0 tx=0x7f2d18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:10.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.112+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2d0c0d2020 con 0x7f2d1c102790 2023-12-11T19:38:10.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.327+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d1c10cf20 con 0x7f2d1c102790 2023-12-11T19:38:10.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.328+0000 7f2d127fc700 1 -- 172.21.15.66:0/3009415814 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d0c0964f0 con 0x7f2d1c102790 2023-12-11T19:38:10.327 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2d080a1500 msgr2=0x7f2d080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2d080a1500 0x7f2d080a39c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1025f0 tx=0x7f2d18009260 comp rx=0 tx=0).stop 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 msgr2=0x7f2d1c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 0x7f2d1c0739c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9a0 tx=0x7f2d0c00ecb0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 shutdown_connections 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f2d080a1500 0x7f2d080a39c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d1c1015a0 0x7f2d1c075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1c103130 0x7f2d1c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 --2- 172.21.15.66:0/3009415814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d1c102790 0x7f2d1c0739c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 >> 172.21.15.66:0/3009415814 conn(0x7f2d1c0fcb10 msgr2=0x7f2d1c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:10.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 shutdown_connections 2023-12-11T19:38:10.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.330+0000 7f2d237ff700 1 -- 172.21.15.66:0/3009415814 wait complete. 2023-12-11T19:38:10.350 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 -- 172.21.15.66:0/1538106919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4103130 msgr2=0x7f3ac41099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 --2- 172.21.15.66:0/1538106919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4103130 0x7f3ac41099e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00b0d0 tx=0x7f3abc00b3e0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 -- 172.21.15.66:0/1538106919 shutdown_connections 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 --2- 172.21.15.66:0/1538106919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4103130 0x7f3ac41099e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 --2- 172.21.15.66:0/1538106919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 0x7f3ac41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 --2- 172.21.15.66:0/1538106919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4102790 0x7f3ac4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 -- 172.21.15.66:0/1538106919 >> 172.21.15.66:0/1538106919 conn(0x7f3ac40fcb10 msgr2=0x7f3ac40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:10.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 -- 172.21.15.66:0/1538106919 shutdown_connections 2023-12-11T19:38:10.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.561+0000 7f3acbbff700 1 -- 172.21.15.66:0/1538106919 wait complete. 2023-12-11T19:38:10.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.562+0000 7f3acbbff700 1 Processor -- start 2023-12-11T19:38:10.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.562+0000 7f3acbbff700 1 -- start start 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.562+0000 7f3acbbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 0x7f3ac4075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.562+0000 7f3acbbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 0x7f3ac40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3acbbff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4103130 0x7f3ac4073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3acbbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40746a0 con 0x7f3ac4102790 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3acbbff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4074810 con 0x7f3ac4103130 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3acbbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41a1590 con 0x7f3ac41015a0 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 0x7f3ac40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 0x7f3ac40739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42786/0 (socket says 172.21.15.66:42786) 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 -- 172.21.15.66:0/3891600330 learned_addr learned my addr 172.21.15.66:0/3891600330 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac999b700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 0x7f3ac4075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 -- 172.21.15.66:0/3891600330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 msgr2=0x7f3ac4075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3aca19c700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4103130 0x7f3ac4073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 0x7f3ac4075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 -- 172.21.15.66:0/3891600330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4103130 msgr2=0x7f3ac4073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4103130 0x7f3ac4073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac919a700 1 -- 172.21.15.66:0/3891600330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc009d00 con 0x7f3ac4102790 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3ac999b700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 0x7f3ac4075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.563+0000 7f3aca19c700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4103130 0x7f3ac4073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.564+0000 7f3ac919a700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 0x7f3ac40739c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400e9a0 tx=0x7f3ab400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.564+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4009790 con 0x7f3ac4102790 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.564+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ab4010430 con 0x7f3ac4102790 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.564+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4018680 con 0x7f3ac4102790 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.564+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac41a1820 con 0x7f3ac4102790 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.564+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac41a1d30 con 0x7f3ac4102790 2023-12-11T19:38:10.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.565+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac4108a20 con 0x7f3ac4102790 2023-12-11T19:38:10.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.567+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3ab40105a0 con 0x7f3ac4102790 2023-12-11T19:38:10.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.569+0000 7f3abaffd700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3ab00a1500 0x7f3ab00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:10.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.570+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3ab4014070 con 0x7f3ac4102790 2023-12-11T19:38:10.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.570+0000 7f3ac999b700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3ab00a1500 0x7f3ab00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:10.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.570+0000 7f3ac999b700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3ab00a1500 0x7f3ab00a39c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ac41025f0 tx=0x7f3ac0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:10.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.574+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3ab40522f0 con 0x7f3ac4102790 2023-12-11T19:38:10.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.789+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ac410cf20 con 0x7f3ac4102790 2023-12-11T19:38:10.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.790+0000 7f3abaffd700 1 -- 172.21.15.66:0/3891600330 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ab40965e0 con 0x7f3ac4102790 2023-12-11T19:38:10.790 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-12-11T19:38:10.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3ab00a1500 msgr2=0x7f3ab00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3ab00a1500 0x7f3ab00a39c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ac41025f0 tx=0x7f3ac0009260 comp rx=0 tx=0).stop 2023-12-11T19:38:10.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 msgr2=0x7f3ac40739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:10.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 0x7f3ac40739c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400e9a0 tx=0x7f3ab400ed60 comp rx=0 tx=0).stop 2023-12-11T19:38:10.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 shutdown_connections 2023-12-11T19:38:10.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7f3ab00a1500 0x7f3ab00a39c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ac4103130 0x7f3ac4073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ac41015a0 0x7f3ac4075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 --2- 172.21.15.66:0/3891600330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ac4102790 0x7f3ac40739c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:10.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 >> 172.21.15.66:0/3891600330 conn(0x7f3ac40fcb10 msgr2=0x7f3ac4105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:10.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 shutdown_connections 2023-12-11T19:38:10.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:10.792+0000 7f3acbbff700 1 -- 172.21.15.66:0/3891600330 wait complete. 2023-12-11T19:38:10.811 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-11T19:38:10.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:10 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1392686315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:38:10.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:10 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3009415814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:38:10.873 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:10 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1392686315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:38:10.873 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:10 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3009415814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:38:11.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 -- 172.21.15.66:0/3662171447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 msgr2=0x7faa20102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:11.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 --2- 172.21.15.66:0/3662171447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20102bf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7faa080099c0 tx=0x7faa08009cd0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 -- 172.21.15.66:0/3662171447 shutdown_connections 2023-12-11T19:38:11.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 --2- 172.21.15.66:0/3662171447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa20103130 0x7faa201099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 --2- 172.21.15.66:0/3662171447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 0x7faa201019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 --2- 172.21.15.66:0/3662171447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20102bf0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 -- 172.21.15.66:0/3662171447 >> 172.21.15.66:0/3662171447 conn(0x7faa200fcb10 msgr2=0x7faa200fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:11.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 -- 172.21.15.66:0/3662171447 shutdown_connections 2023-12-11T19:38:11.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.020+0000 7faa24fff700 1 -- 172.21.15.66:0/3662171447 wait complete. 2023-12-11T19:38:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:10 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1392686315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T19:38:11.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:10 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3009415814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T19:38:11.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.021+0000 7faa24fff700 1 Processor -- start 2023-12-11T19:38:11.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.021+0000 7faa24fff700 1 -- start start 2023-12-11T19:38:11.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa24fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 0x7faa20199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa24fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa24fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa20103130 0x7faa201a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa24fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201a1120 con 0x7faa20102790 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa24fff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201a1260 con 0x7faa20103130 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa24fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201a13f0 con 0x7faa201015a0 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42808/0 (socket says 172.21.15.66:42808) 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 -- 172.21.15.66:0/1324669586 learned_addr learned my addr 172.21.15.66:0/1324669586 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa1ed9d700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa20103130 0x7faa201a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa1e59c700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 0x7faa20199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 -- 172.21.15.66:0/1324669586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 msgr2=0x7faa20199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 0x7faa20199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 -- 172.21.15.66:0/1324669586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa20103130 msgr2=0x7faa201a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:11.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa20103130 0x7faa201a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa17fff700 1 -- 172.21.15.66:0/1324669586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080095f0 con 0x7faa20102790 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.022+0000 7faa1e59c700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 0x7faa20199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.023+0000 7faa17fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20199b80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7faa08005910 tx=0x7faa08005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.023+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0801d070 con 0x7faa20102790 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.023+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa0800bd30 con 0x7faa20102790 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.023+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08022dc0 con 0x7faa20102790 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.023+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa201a1670 con 0x7faa20102790 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.023+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa201a1a40 con 0x7faa20102790 2023-12-11T19:38:11.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.024+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20068310 con 0x7faa20102790 2023-12-11T19:38:11.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.028+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 136006+0+0 (secure 0 0 0) 0x7faa08004010 con 0x7faa20102790 2023-12-11T19:38:11.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.029+0000 7faa177fe700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faa0c0a1500 0x7faa0c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T19:38:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.029+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faa080ce030 con 0x7faa20102790 2023-12-11T19:38:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.029+0000 7faa1e59c700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faa0c0a1500 0x7faa0c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T19:38:11.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.030+0000 7faa1e59c700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faa0c0a1500 0x7faa0c0a39c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa10005fd0 tx=0x7faa10005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T19:38:11.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.031+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faa08097720 con 0x7faa20102790 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.263+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa2010ccc0 con 0x7faa20102790 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.263+0000 7faa177fe700 1 -- 172.21.15.66:0/1324669586 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+590 (secure 0 0 0) 0x7faa08027260 con 0x7faa20102790 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/2642090748 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/4268281106 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/153702681 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1199617033 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1568045414 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/294726661 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3525772822 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:11.263 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3525772822 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:11.264 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1790110256 2023-12-12T19:23:41.756564+0000 2023-12-11T19:38:11.264 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/4268281106 2023-12-12T19:24:10.190965+0000 2023-12-11T19:38:11.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faa0c0a1500 msgr2=0x7faa0c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faa0c0a1500 0x7faa0c0a39c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa10005fd0 tx=0x7faa10005f00 comp rx=0 tx=0).stop 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 msgr2=0x7faa20199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20199b80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7faa08005910 tx=0x7faa08005b00 comp rx=0 tx=0).stop 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 shutdown_connections 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:6800/775613503,v1:172.21.15.66:6801/775613503] conn(0x7faa0c0a1500 0x7faa0c0a39c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa20103130 0x7faa201a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa201015a0 0x7faa20199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 --2- 172.21.15.66:0/1324669586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa20102790 0x7faa20199b80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 >> 172.21.15.66:0/1324669586 conn(0x7faa200fcb10 msgr2=0x7faa20073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 shutdown_connections 2023-12-11T19:38:11.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-12-11T19:38:11.266+0000 7faa24fff700 1 -- 172.21.15.66:0/1324669586 wait complete. 2023-12-11T19:38:11.267 INFO:teuthology.orchestra.run.smithi066.stderr:listed 10 entries 2023-12-11T19:38:11.285 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2023-12-11T19:38:11.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:38:11.285 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-11T19:38:11.362 INFO:teuthology.orchestra.run:waiting for 300 2023-12-11T19:38:11.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi066.stderr:ceph-fuse[158721]: fuse finished with error 0 and tester_r 0 2023-12-11T19:38:11.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:11 smithi070 ceph-mon[104044]: pgmap v498: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:38:11.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:11 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/3891600330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:38:11.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:11 smithi070 ceph-mon[104044]: from='client.? 172.21.15.66:0/1324669586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T19:38:11.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:11 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:38:12.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:11 smithi167 ceph-mon[102750]: pgmap v498: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:38:12.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:11 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/3891600330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:38:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:11 smithi167 ceph-mon[102750]: from='client.? 172.21.15.66:0/1324669586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T19:38:12.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:11 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:38:12.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:11 smithi066 ceph-mon[97920]: pgmap v498: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-11T19:38:12.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:11 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/3891600330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T19:38:12.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:11 smithi066 ceph-mon[97920]: from='client.? 172.21.15.66:0/1324669586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T19:38:12.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:11 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T19:38:12.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:12 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:38:12.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:12 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:38:12.860 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:12 smithi070 ceph-mon[104044]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:38:13.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:12 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:38:13.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:12 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:38:13.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:12 smithi167 ceph-mon[102750]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:38:13.061 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T19:38:13.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T19:38:13.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:12 smithi066 ceph-mon[97920]: from='mgr.14150 172.21.15.66:0/106528740' entity='mgr.x' 2023-12-11T19:38:14.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:13 smithi167 ceph-mon[102750]: pgmap v499: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:14.061 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:13 smithi066 ceph-mon[97920]: pgmap v499: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:14.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:13 smithi070 ceph-mon[104044]: pgmap v499: 97 pgs: 97 active+clean; 1005 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-11T19:38:16.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:15 smithi167 ceph-mon[102750]: pgmap v500: 97 pgs: 97 active+clean; 1006 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-11T19:38:16.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:15 smithi066 ceph-mon[97920]: pgmap v500: 97 pgs: 97 active+clean; 1006 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-11T19:38:16.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:15 smithi070 ceph-mon[104044]: pgmap v500: 97 pgs: 97 active+clean; 1006 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-11T19:38:17.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:38:17.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:38:17.464 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-11T19:38:17.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:38:17.528 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-12-11T19:38:17.576 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-11T19:38:17.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:38:17.577 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T19:38:17.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T19:38:17.652 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2023-12-11T19:38:17.755 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-11T19:38:17.767 INFO:tasks.cephadm:Teardown begin 2023-12-11T19:38:17.767 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/cephadm.py", line 1794, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-12-11T19:38:17.768 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:38:17.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:38:17.913 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:17 smithi070 ceph-mon[104044]: pgmap v501: 97 pgs: 97 active+clean; 1006 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2023-12-11T19:38:17.923 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:38:17.992 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:17 smithi167 ceph-mon[102750]: pgmap v501: 97 pgs: 97 active+clean; 1006 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2023-12-11T19:38:18.003 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-11T19:38:18.003 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-11T19:38:18.049 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-11T19:38:18.062 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:17 smithi066 ceph-mon[97920]: pgmap v501: 97 pgs: 97 active+clean; 1006 KiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2023-12-11T19:38:18.094 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-11T19:38:18.139 INFO:tasks.cephadm:Stopping all daemons... 2023-12-11T19:38:18.140 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-11T19:38:18.140 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a 2023-12-11T19:38:18.562 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:18 smithi066 systemd[1]: Stopping Ceph mon.a for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:19.020 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a[97896]: 2023-12-11T19:38:18.679+0000 7efd87de3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:19.020 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a[97896]: 2023-12-11T19:38:18.679+0000 7efd87de3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-11T19:38:19.311 INFO:journalctl@ceph.mon.a.smithi066.stdout:Dec 11 19:38:19 smithi066 bash[166062]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-a 2023-12-11T19:38:19.523 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.a.service' 2023-12-11T19:38:19.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:19.627 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-11T19:38:19.627 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-11T19:38:19.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.b 2023-12-11T19:38:20.109 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:19 smithi070 systemd[1]: Stopping Ceph mon.b for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:20.516 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-b[104020]: 2023-12-11T19:38:20.163+0000 7f71df4b3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:20.516 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-b[104020]: 2023-12-11T19:38:20.163+0000 7f71df4b3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-11T19:38:20.859 INFO:journalctl@ceph.mon.b.smithi070.stdout:Dec 11 19:38:20 smithi070 bash[143678]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-b 2023-12-11T19:38:20.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.b.service' 2023-12-11T19:38:21.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:21.030 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-11T19:38:21.030 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-11T19:38:21.030 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.c 2023-12-11T19:38:21.519 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:21 smithi167 systemd[1]: Stopping Ceph mon.c for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:22.013 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-c[102726]: 2023-12-11T19:38:21.553+0000 7f038949e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:22.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-c[102726]: 2023-12-11T19:38:21.553+0000 7f038949e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-11T19:38:22.269 INFO:journalctl@ceph.mon.c.smithi167.stdout:Dec 11 19:38:22 smithi167 bash[143456]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mon-c 2023-12-11T19:38:22.496 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mon.c.service' 2023-12-11T19:38:22.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:22.594 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-11T19:38:22.594 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-11T19:38:22.594 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x 2023-12-11T19:38:23.062 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:38:22 smithi066 systemd[1]: Stopping Ceph mgr.x for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:23.384 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:38:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:38:23.120+0000 7f9fcf4aa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:23.384 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:38:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x[98169]: 2023-12-11T19:38:23.120+0000 7f9fcf4aa700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-11T19:38:23.811 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Dec 11 19:38:23 smithi066 bash[166211]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-x 2023-12-11T19:38:23.857 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.x.service' 2023-12-11T19:38:23.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:23.948 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-11T19:38:23.948 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-11T19:38:23.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.y 2023-12-11T19:38:24.359 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:38:24 smithi070 systemd[1]: Stopping Ceph mgr.y for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:25.109 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Dec 11 19:38:24 smithi070 bash[143828]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-y 2023-12-11T19:38:25.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.y.service' 2023-12-11T19:38:25.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:25.341 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-11T19:38:25.341 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-11T19:38:25.341 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.z 2023-12-11T19:38:25.769 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:38:25 smithi167 systemd[1]: Stopping Ceph mgr.z for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:26.433 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:38:26 smithi167 bash[143615]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mgr-z 2023-12-11T19:38:26.662 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.z.service' 2023-12-11T19:38:26.722 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:38:26 smithi167 systemd[1]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-11T19:38:26.722 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:38:26 smithi167 systemd[1]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2023-12-11T19:38:26.722 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Dec 11 19:38:26 smithi167 systemd[1]: Stopped Ceph mgr.z for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:38:26.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:26.750 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-11T19:38:26.750 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-11T19:38:26.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.0 2023-12-11T19:38:27.279 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:38:26 smithi066 systemd[1]: Stopping Ceph osd.0 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:27.561 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:38:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-0[112658]: 2023-12-11T19:38:27.278+0000 7f1e1b191700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:27.562 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:38:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-0[112658]: 2023-12-11T19:38:27.278+0000 7f1e1b191700 -1 osd.0 88 *** Got signal Terminated *** 2023-12-11T19:38:27.562 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:38:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-0[112658]: 2023-12-11T19:38:27.278+0000 7f1e1b191700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:38:33.224 INFO:journalctl@ceph.osd.0.smithi066.stdout:Dec 11 19:38:33 smithi066 bash[166359]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-0 2023-12-11T19:38:35.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.0.service' 2023-12-11T19:38:35.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:35.124 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-11T19:38:35.124 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-11T19:38:35.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.1 2023-12-11T19:38:35.609 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:38:35 smithi070 systemd[1]: Stopping Ceph osd.1 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:36.109 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:38:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-1[109163]: 2023-12-11T19:38:35.661+0000 7fcbb7f70700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:36.109 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:38:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-1[109163]: 2023-12-11T19:38:35.661+0000 7fcbb7f70700 -1 osd.1 88 *** Got signal Terminated *** 2023-12-11T19:38:36.109 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:38:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-1[109163]: 2023-12-11T19:38:35.661+0000 7fcbb7f70700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:38:41.338 INFO:journalctl@ceph.osd.1.smithi070.stdout:Dec 11 19:38:41 smithi070 bash[143980]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-1 2023-12-11T19:38:43.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.1.service' 2023-12-11T19:38:43.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:43.284 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-11T19:38:43.285 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-11T19:38:43.285 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.2 2023-12-11T19:38:43.770 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:38:43 smithi167 systemd[1]: Stopping Ceph osd.2 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:44.269 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:38:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-2[108594]: 2023-12-11T19:38:43.792+0000 7f494b31c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:44.269 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:38:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-2[108594]: 2023-12-11T19:38:43.792+0000 7f494b31c700 -1 osd.2 88 *** Got signal Terminated *** 2023-12-11T19:38:44.269 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:38:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-2[108594]: 2023-12-11T19:38:43.792+0000 7f494b31c700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:38:49.769 INFO:journalctl@ceph.osd.2.smithi167.stdout:Dec 11 19:38:49 smithi167 bash[143768]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-2 2023-12-11T19:38:51.552 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.2.service' 2023-12-11T19:38:51.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:51.644 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-11T19:38:51.644 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-11T19:38:51.644 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.3 2023-12-11T19:38:52.062 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:38:51 smithi066 systemd[1]: Stopping Ceph osd.3 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:38:52.561 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:38:52 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:38:52.166+0000 7f3508a95700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:38:52.562 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:38:52 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:38:52.166+0000 7f3508a95700 -1 osd.3 88 *** Got signal Terminated *** 2023-12-11T19:38:52.562 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:38:52 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:38:52.166+0000 7f3508a95700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:38:53.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:53 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:53.450+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:54.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:54 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:54.447+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:55.117 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:38:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:38:54.784+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:38:55.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:38:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:38:55.118+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:38:55.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:55 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:55.478+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:56.109 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:38:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:38:55.772+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:38:56.109 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:38:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:38:55.991+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:38:56.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:38:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:38:56.109+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:38:56.812 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:38:56 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3[121094]: 2023-12-11T19:38:56.379+0000 7f350488f700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.441064+0000 front 2023-12-11T19:38:30.440975+0000 (oldest deadline 2023-12-11T19:38:56.341199+0000) 2023-12-11T19:38:56.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:56 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:56.469+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:57.081 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:38:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:38:56.791+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:38:57.082 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:38:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:38:56.955+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:38:57.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:38:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:38:57.081+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:38:57.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:57 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:57.514+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:57.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:38:57 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:38:57.567+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:38:58.088 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:38:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:38:57.795+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:38:58.088 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:38:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:38:57.973+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:38:58.313 INFO:journalctl@ceph.osd.3.smithi066.stdout:Dec 11 19:38:57 smithi066 bash[166646]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-3 2023-12-11T19:38:58.313 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:38:57 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:38:57.872+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:38:58.521 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:38:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:38:58.089+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:38:58.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:58.466+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:58.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:38:58 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:38:58.587+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:38:59.046 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:38:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:38:58.793+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:38:59.046 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:38:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:38:59.015+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:38:59.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:38:58 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:38:58.913+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:38:59.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:38:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:38:59.046+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:38:59.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:38:59 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:38:59.418+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:38:59.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:38:59 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:38:59.566+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:38:59.903 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.3.service' 2023-12-11T19:38:59.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:38:59.998 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-11T19:38:59.998 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-11T19:38:59.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.4 2023-12-11T19:39:00.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:00.015+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:00.269 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:38:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:38:59.834+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:00.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:00.042+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:00.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:38:59 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:38:59.920+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:00.359 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:00 smithi070 systemd[1]: Stopping Ceph osd.4 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:00.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:00 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:00.420+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:00.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:00 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:00.585+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:00.859 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:00 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:39:00.528+0000 7f8659569700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:00.859 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:00 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:39:00.528+0000 7f8659569700 -1 osd.4 88 *** Got signal Terminated *** 2023-12-11T19:39:00.859 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:00 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:39:00.528+0000 7f8659569700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:01.088 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:00.788+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:01.088 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:01.031+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:01.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:00 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:00.966+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:01.087+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:01.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:01.426+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:02.075 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:01.777+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:02.114 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:01 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:01.618+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:02.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:01.963+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:02.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:01 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:01.963+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:02.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:02.075+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:02.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:02.107+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:02.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:02.107+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:02.814 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:02.387+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:03.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:02.777+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:03.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:02 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:02.657+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:03.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:02 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:02.657+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:03.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:02.929+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:03.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:02 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:02.929+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:03.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:03.083+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:03.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:03.116+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:03.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:03.116+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:03.609 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:03 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:39:03.406+0000 7f8654361700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.516704+0000 front 2023-12-11T19:38:39.516741+0000 (oldest deadline 2023-12-11T19:39:03.016972+0000) 2023-12-11T19:39:03.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:03 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:03.233+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:03.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:03 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:03.384+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:04.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:03.739+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:04.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:03 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:03.675+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:04.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:03 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:03.675+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:04.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:03 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:03.975+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:04.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:03 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:03.975+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:04.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:04 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:04.093+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:04.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:04 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:04.093+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:04.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:04 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:04.095+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:04.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:04 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:04.095+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:04.609 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:04 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:39:04.398+0000 7f8654361700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.516704+0000 front 2023-12-11T19:38:39.516741+0000 (oldest deadline 2023-12-11T19:39:03.016972+0000) 2023-12-11T19:39:04.610 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:04 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:04.202+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:04.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:04 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:04.419+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:04.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:04 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:04.419+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:05.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:04 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:04.711+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:05.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:04 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:04.711+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:05.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:04 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:04.654+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:05.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:04 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:04.654+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:05.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:05.023+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:05.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:05.023+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:05.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:05 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:05.139+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:05.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:05 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:05.139+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:05.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:05 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:05.047+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:05.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:05 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:05.047+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:05.609 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:05 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4[117177]: 2023-12-11T19:39:05.352+0000 7f8654361700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.516704+0000 front 2023-12-11T19:38:39.516741+0000 (oldest deadline 2023-12-11T19:39:03.016972+0000) 2023-12-11T19:39:05.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:05 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:05.201+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:05.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:05.393+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:05.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:05 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:05.393+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:06.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:05 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:05.720+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:06.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:05 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:05.720+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:06.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:05 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:05.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:06.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:05 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:05.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:06.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:06.013+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:06.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:06.013+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:06.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:06.048+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:06.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:06.048+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:06.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:06.097+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:06.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:06.097+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:06.609 INFO:journalctl@ceph.osd.4.smithi070.stdout:Dec 11 19:39:06 smithi070 bash[144252]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-4 2023-12-11T19:39:06.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:06 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:06.214+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:06.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:06.405+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:06.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:06.406+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:07.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:06.758+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:07.020 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:06.758+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:07.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:06 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:06.666+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:07.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:06 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:06.666+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:07.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:07 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:07.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:07.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:07 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:07.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:07.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:07 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:07.076+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:07.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:07 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:07.076+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:07.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:07 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:07.131+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:07.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:07 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:07.131+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:07.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:07 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:07.218+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:07.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:07 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:07.385+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:07.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:07 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:07.385+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:07.941 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:07 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:07.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:07.941 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:07 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:07.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:08.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:07 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:07.721+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:08.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:07 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:07.721+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:08.290 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:08 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:08.170+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:08.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.4.service' 2023-12-11T19:39:08.378 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:08.079+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:08.378 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:08.079+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:08.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:08.409 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-11T19:39:08.409 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-11T19:39:08.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.5 2023-12-11T19:39:08.477 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:08.131+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:08.478 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:08.131+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:08.478 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:08.082+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:08.478 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:08.082+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:08.769 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:08 smithi167 systemd[1]: Stopping Ceph osd.5 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:08.770 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:08.720+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:08.770 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:08.720+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:08.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:08.378+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:08.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:08 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:08.378+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:08.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:08 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:08.574+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:08.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:08 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:08.574+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:09.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:09.125+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:09.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:09.125+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:09.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:09.047+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:09.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:09.047+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:09.271 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:08.929+0000 7f32b1eb5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:09.271 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:08.929+0000 7f32b1eb5700 -1 osd.5 88 *** Got signal Terminated *** 2023-12-11T19:39:09.271 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:08 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:08.929+0000 7f32b1eb5700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:09.396 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:09 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:09.109+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:09.397 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:09 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:09.109+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:09.546 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:09 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:09.210+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:09.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:09 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:09.397+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:09.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:09 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:09.397+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:09.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:09 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:09.547+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:09.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:09 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:09.547+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:10.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:09.698+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:10.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:09.698+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:10.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:09 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:09.698+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.186621+0000 front 2023-12-11T19:38:48.186651+0000 (oldest deadline 2023-12-11T19:39:09.286766+0000) 2023-12-11T19:39:10.383 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:10 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:10.106+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:10.383 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:10 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:10.106+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:10.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:10.054+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:10.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:10.054+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:10.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:10.128+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:10.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:10.128+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:10.593 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:10 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:10.229+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:10.814 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:10 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:10.383+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:10.814 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:10 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:10.383+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:10.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:10 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:10.594+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:10.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:10 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:10.594+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:11.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:10.740+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:11.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:10.740+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:11.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:10 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:10.740+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.186621+0000 front 2023-12-11T19:38:48.186651+0000 (oldest deadline 2023-12-11T19:39:09.286766+0000) 2023-12-11T19:39:11.340 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:11 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:11.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:11.340 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:11 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:11.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:11.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:11.094+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:11.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:11.094+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:11.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:11.090+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:11.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:11.090+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:11.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:11.090+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:11.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:11 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:11.231+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:11.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:11 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:11.231+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:11.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:11 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:11.341+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:11.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:11 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:11.341+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:12.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:11.714+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:12.020 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:11.714+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:12.020 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:11 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:11.714+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.186621+0000 front 2023-12-11T19:38:48.186651+0000 (oldest deadline 2023-12-11T19:39:09.286766+0000) 2023-12-11T19:39:12.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:11 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:11.639+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:12.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:11 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:11.639+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:12.389 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:12.069+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:12.389 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:12.069+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:12.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:12.110+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:12.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:12.110+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:12.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:12.139+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:12.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:12.139+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:12.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:12.139+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:12.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:12 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:12.240+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:12.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:12 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:12.240+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:12.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:12.389+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:12.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:12.389+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:12.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:12.389+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:13.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:12.687+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:13.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:12.687+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:13.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:12 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:12.687+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.186621+0000 front 2023-12-11T19:38:48.186651+0000 (oldest deadline 2023-12-11T19:39:09.286766+0000) 2023-12-11T19:39:13.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:12 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:12.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:13.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:12 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:12.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:13.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:12 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:12.620+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:13.410 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:13.079+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:13.410 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:13.079+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:13.410 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:13.079+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:13.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:13.091+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:13.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:13.091+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:13.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:13.091+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:13.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:13.072+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:13.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:13.072+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:13.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:13.072+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:13.584 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:13 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:13.205+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:13.584 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:13 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:13.205+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:13.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:13.409+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:13.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:13.409+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:13.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:13 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:13.409+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:13.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:13 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:13.584+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:13.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:13 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:13.584+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:13.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:13 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:13.584+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:14.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:13.696+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.784231+0000 front 2023-12-11T19:38:28.784417+0000 (oldest deadline 2023-12-11T19:38:54.084162+0000) 2023-12-11T19:39:14.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:13.696+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.785808+0000 front 2023-12-11T19:38:38.785917+0000 (oldest deadline 2023-12-11T19:39:04.685603+0000) 2023-12-11T19:39:14.019 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:13 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5[117021]: 2023-12-11T19:39:13.696+0000 7f32abcab700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.186621+0000 front 2023-12-11T19:38:48.186651+0000 (oldest deadline 2023-12-11T19:39:09.286766+0000) 2023-12-11T19:39:14.429 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:14 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:14.082+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:14.429 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:14 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:14.082+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:14.429 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:14 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:14.082+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:14.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:14 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:14.113+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:14.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:14 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:14.113+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:14.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:14 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:14.113+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:14.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:14 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:14.036+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:14.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:14 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:14.036+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:14.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:14 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:14.036+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:14.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:14 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:14.184+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:14.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:14 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:14.184+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:14.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:14 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:14.428+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:14.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:14 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:14.428+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:14.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:14 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:14.428+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:15.007 INFO:journalctl@ceph.osd.5.smithi167.stdout:Dec 11 19:39:14 smithi167 bash[144051]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-5 2023-12-11T19:39:15.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:14 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:14.624+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:15.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:14 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:14.624+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:15.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:14 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:14.624+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:15.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:15 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:15.118+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:15.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:15 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:15.118+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:15.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:15 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:15.118+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:15.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:15 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:15.006+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:15.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:15 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:15.006+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:15.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:15 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:15.006+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:15.453 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:15 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:15.127+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:15.453 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:15 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:15.127+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:15.453 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:15 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:15.127+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:15.597 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:15 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:15.160+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:15.597 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:15 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:15.160+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:15.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:15 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:15.453+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:15.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:15 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:15.453+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:15.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:15 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:15.453+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:15.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:15 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:15.598+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:15.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:15 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:15.598+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:15.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:15 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:15.598+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:16.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:16.125+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:16.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:16.125+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:16.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:16.125+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:16.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:16.006+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:16.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:16.006+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:16.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:16.006+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:16.404 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:16.094+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:16.405 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:16.094+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:16.405 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:16.094+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:16.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.5.service' 2023-12-11T19:39:16.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:16 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:16.144+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:16.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:16 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:16.144+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:16.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:16.657 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-11T19:39:16.657 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-11T19:39:16.657 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.6 2023-12-11T19:39:16.727 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:16.404+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:16.727 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:16.404+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:16.727 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:16 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:16.404+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:17.055 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:16 smithi066 systemd[1]: Stopping Ceph osd.6 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:17.094 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:16 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:16.636+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:17.094 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:16 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:16.636+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:17.095 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:16 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:16.636+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:17.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:17 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:17.082+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:17.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:17 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:17.082+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:17.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:17 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:17.082+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:17.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:16.986+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:17.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:16.986+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:17.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:16 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:16.986+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:17.312 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:17.203+0000 7f0cd5f93700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:17.312 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:17.203+0000 7f0cd5f93700 -1 osd.6 88 *** Got signal Terminated *** 2023-12-11T19:39:17.312 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:17.203+0000 7f0cd5f93700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:17.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:17.054+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:17.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:17.054+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:17.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:17.054+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:17.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:17 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:17.094+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:17.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:17 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:17.094+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:17.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:17.439+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:17.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:17.439+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:17.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:17.439+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:18.102 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:17 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:17.629+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:18.102 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:17 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:17.629+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:18.102 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:17 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:17.629+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:18.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:18 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:18.101+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:18.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:18 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:18.101+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:18.399 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:18.065+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:18.399 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:18.065+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:18.400 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:18.065+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:18.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:18 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:18.070+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:18.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:18 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:18.070+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:18.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:18 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:18.070+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:18.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:18 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:18.023+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:18.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:18 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:18.023+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:18.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:18 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:18.023+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:18.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:18.398+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:18.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:18.398+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:18.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:18 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:18.398+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:19.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:18 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:18.652+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:19.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:18 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:18.652+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:19.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:18 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:18.652+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:19.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:18 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:18.652+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:19.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:19.048+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:19.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:19.048+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:19.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:19.048+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:19.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:19.048+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:19.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:19 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:19.058+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:19.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:19 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:19.058+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:19.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:19 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:19.058+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:19.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:19 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:19.051+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:19.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:19 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:19.051+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:19.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:19 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:19.051+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:19.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:19.148+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:19.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:19.148+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:19.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:19.148+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:19.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:19.418+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:19.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:19.418+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:19.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:19.418+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:19.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:19 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:19.418+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.993097+0000 front 2023-12-11T19:38:54.993034+0000 (oldest deadline 2023-12-11T19:39:19.093009+0000) 2023-12-11T19:39:20.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:19.635+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:20.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:19.635+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:20.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:19.635+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:20.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:19 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:19.635+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:20.387 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:20.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:20.387 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:20.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:20.387 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:20.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:20.387 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:20.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:20.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:20.107+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:20.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:20.107+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:20.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:20.107+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:20.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:20.107+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:20.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:20.087+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:20.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:20.087+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:20.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:20 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:20.087+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:20.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:20.181+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:20.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:20.181+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:20.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:20.181+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:20.811 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:20.387+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:20.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:20.387+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:20.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:20.387+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:20.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:20.387+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.993097+0000 front 2023-12-11T19:38:54.993034+0000 (oldest deadline 2023-12-11T19:39:19.093009+0000) 2023-12-11T19:39:21.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:20.609+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:21.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:20.609+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:21.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:20.609+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:21.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:20 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:20.609+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:21.402 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:21.099+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:21.402 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:21.099+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:21.402 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:21.099+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:21.403 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:21.099+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:21.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:21.063+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:21.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:21.063+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:21.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:21.063+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:21.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:21.063+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:21.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:21.090+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:21.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:21.090+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:21.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:21 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:21.090+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:21.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:21.184+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:21.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:21.184+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:21.609 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:21.184+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:21.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:21.402+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:28.187998+0000 front 2023-12-11T19:38:28.188024+0000 (oldest deadline 2023-12-11T19:38:52.789300+0000) 2023-12-11T19:39:21.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:21.402+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.590645+0000 front 2023-12-11T19:38:38.590906+0000 (oldest deadline 2023-12-11T19:39:03.890724+0000) 2023-12-11T19:39:21.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:21.402+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:43.891609+0000 front 2023-12-11T19:38:43.891656+0000 (oldest deadline 2023-12-11T19:39:11.492136+0000) 2023-12-11T19:39:21.812 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:21 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6[129569]: 2023-12-11T19:39:21.402+0000 7f0ccfd89700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.993097+0000 front 2023-12-11T19:38:54.993034+0000 (oldest deadline 2023-12-11T19:39:19.093009+0000) 2023-12-11T19:39:22.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:21.634+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:22.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:21.634+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:22.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:21.634+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:22.109 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:21 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:21.634+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:22.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.026+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:22.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.026+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:22.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.026+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:22.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.026+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:22.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:22.059+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:22.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:22.059+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:22.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:22.059+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:22.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:22.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:22.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:22.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:22.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:22.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:22.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:22 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:22.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:22.590 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:22.180+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:22.590 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:22.180+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:22.590 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:22.180+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:22.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:22.589+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:22.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:22.589+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:22.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:22.589+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:22.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:22 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:22.589+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:23.128 INFO:journalctl@ceph.osd.6.smithi066.stdout:Dec 11 19:39:22 smithi066 bash[166917]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-6 2023-12-11T19:39:23.128 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:23.114+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:23.128 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:23.114+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:23.128 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:23.114+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:23.128 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:23 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:23.114+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:23.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.995+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:23.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.995+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:23.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.995+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:23.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:22 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:22.995+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:23.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:23 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:23.067+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:23.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:23 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:23.067+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:23.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:23 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:23.067+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:23.574 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:23.144+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:23.574 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:23.144+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:23.574 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:23.144+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:23.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:23.574+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:23.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:23.574+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:23.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:23.574+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:23.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:23 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:23.574+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:24.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:24.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:24.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:24.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:24.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:24.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:24.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:24 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:24.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:24.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:24.031+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:24.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:24.031+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:24.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:24.031+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:24.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:24.031+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:24.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:24.116+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:24.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:24.116+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:24.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:24 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:24.116+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:24.535 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:24.148+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:24.535 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:24.148+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:24.535 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:24.148+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:24.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:24.535+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:24.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:24.535+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:24.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:24.535+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:24.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:24 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:24.535+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:24.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.6.service' 2023-12-11T19:39:24.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:24.993 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-11T19:39:24.993 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-11T19:39:24.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.7 2023-12-11T19:39:25.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 systemd[1]: Stopping Ceph osd.7 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:25.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:25.112+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:25.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:25.112+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:25.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:25.112+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:25.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:25.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:25.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:25.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:25.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:25.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:25.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:25.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:25.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:25.153+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:25.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:25.153+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:25.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:25 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:25.153+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:25.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:25 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:25.155+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:25.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:25 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:25.155+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:25.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:25 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:25.155+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:25.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:25 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:25.155+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:25.522+0000 7f8f2c33c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:25.522+0000 7f8f2c33c700 -1 osd.7 88 *** Got signal Terminated *** 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:25.522+0000 7f8f2c33c700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:25.561+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:25.561+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:25.561+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:25.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:25 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:25.561+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:26.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:26.085+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:26.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:26.085+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:26.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:26.085+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:26.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:26.044+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:26.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:26.044+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:26.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:26.044+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:26.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:26.044+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:26.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:26.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:26.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:26.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:26.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:26 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:26.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:26.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:26 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:26.189+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:26.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:26 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:26.189+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:26.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:26 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:26.189+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:26.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:26 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:26.189+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:26.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:26.566+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:26.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:26.566+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:26.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:26.566+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:26.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:26.566+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:26.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:26.566+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:27.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:27.079+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:27.360 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:27.079+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:27.360 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:27.079+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:27.360 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:27.079+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.163448+0000 front 2023-12-11T19:39:05.163487+0000 (oldest deadline 2023-12-11T19:39:26.863570+0000) 2023-12-11T19:39:27.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:27.206+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:27.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:27.206+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:27.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:27.206+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:27.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:27.037+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:27.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:27.037+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:27.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:27.037+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:27.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:27 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:27.037+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:27.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:27.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:27.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:27.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:27.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:27.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:27.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:27.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:27.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:27 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:27.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:27.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:27.580+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:27.860 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:27.580+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:27.860 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:27.580+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:27.860 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:27.580+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:27.860 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:27 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:27.580+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:28.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:28.048+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:28.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:28.048+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:28.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:28.048+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:28.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:28.048+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.163448+0000 front 2023-12-11T19:39:05.163487+0000 (oldest deadline 2023-12-11T19:39:26.863570+0000) 2023-12-11T19:39:28.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:28.216+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:28.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:28.216+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:28.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:28.216+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:28.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:28.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:28.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:28.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:28.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:28.060+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:28.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:28.060+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:28.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:28 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:28.060+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:28.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:28 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:28.178+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:28.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:28 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:28.178+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:28.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:28 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:28.178+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:28.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:28 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:28.178+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:28.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:28 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:28.178+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:28.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:28.563+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:28.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:28.563+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:28.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:28.563+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:28.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:28.563+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:28.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:28 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:28.563+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:29.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:29.098+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:29.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:29.098+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:29.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:29.098+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:29.359 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:29.098+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.163448+0000 front 2023-12-11T19:39:05.163487+0000 (oldest deadline 2023-12-11T19:39:26.863570+0000) 2023-12-11T19:39:29.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:29.246+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:29.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:29.246+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:29.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:29.246+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:29.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:29.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:29.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:29.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:29.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:29.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:29.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:29.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:29.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:29 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:29.059+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:29.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:29 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:29.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:29.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:29 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:29.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:29.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:29 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:29.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:29.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:29 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:29.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:29.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:29 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:29.183+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:29.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:29.520+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:29.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:29.520+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:29.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:29.520+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:29.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:29.520+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:29.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:29 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:29.520+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:30.491 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:30.140+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:39.558621+0000 front 2023-12-11T19:38:39.558648+0000 (oldest deadline 2023-12-11T19:39:03.058860+0000) 2023-12-11T19:39:30.491 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:30.140+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:45.960009+0000 front 2023-12-11T19:38:45.959832+0000 (oldest deadline 2023-12-11T19:39:10.660013+0000) 2023-12-11T19:39:30.491 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:30.140+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.161395+0000 front 2023-12-11T19:38:54.161328+0000 (oldest deadline 2023-12-11T19:39:18.861233+0000) 2023-12-11T19:39:30.491 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7[125189]: 2023-12-11T19:39:30.140+0000 7f8f26933700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.163448+0000 front 2023-12-11T19:39:05.163487+0000 (oldest deadline 2023-12-11T19:39:26.863570+0000) 2023-12-11T19:39:30.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:30.202+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:30.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:30.202+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:30.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:30.202+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:30.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:30.067+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:30.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:30.067+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:30.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:30.067+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:30.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:30.067+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:30.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:30 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:30.067+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:30.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:30 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:30.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:30.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:30 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:30.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:30.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:30 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:30.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:30.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:30 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:30.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:30.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:30 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:30.135+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:30.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:30.491+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:30.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:30.491+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:30.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:30.491+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:30.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:30.491+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:30.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:30 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:30.491+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:31.506 INFO:journalctl@ceph.osd.7.smithi070.stdout:Dec 11 19:39:31 smithi070 bash[144526]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-7 2023-12-11T19:39:31.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:31.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:31.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:31.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:31.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:31.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:31.519 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:31.053+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:31.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:31.053+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:31.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:31.053+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:31.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:31.053+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:31.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:31 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:31.053+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:31.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:31 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:31.101+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:31.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:31 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:31.101+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:31.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:31 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:31.101+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:31.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:31 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:31.101+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:31.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:31 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:31.101+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:31.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:31.506+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:31.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:31.506+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:31.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:31.506+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:31.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:31.506+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:31.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:31 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:31.506+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:32.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:32.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:32.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:32.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:32.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:32.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:32.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:32.163+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:32.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:32.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:32.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:32.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:32.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:32.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:32.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:32.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:32.520 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:32 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:32.052+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:32.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:32.084+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:32.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:32.084+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:32.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:32.084+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:32.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:32.084+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:32.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:32 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:32.084+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:32.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:32.486+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:32.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:32.486+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:32.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:32.486+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:32.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:32.486+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:32.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:32 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:32.486+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:33.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.7.service' 2023-12-11T19:39:33.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:33.236 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-11T19:39:33.236 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-11T19:39:33.236 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.8 2023-12-11T19:39:33.304 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.047+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:33.304 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.047+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:33.304 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.047+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:33.304 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.047+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:33.304 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.047+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:33.305 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:33.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:33.305 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:33.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:33.305 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:33.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:33.305 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:33.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:33.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:33.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:33.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:33.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:33.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:33.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:33.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:33.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:33.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:33 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:33.085+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:33.751 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 systemd[1]: Stopping Ceph osd.8 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:33.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:33.468+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:33.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:33.468+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:33.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:33.468+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:33.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:33.468+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:33.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:33 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:33.468+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:34.006 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.751+0000 7ffa1fae8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:34.006 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.751+0000 7ffa1fae8700 -1 osd.8 88 *** Got signal Terminated *** 2023-12-11T19:39:34.007 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:33 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:33.751+0000 7ffa1fae8700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:34.007 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.007+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:34.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.007+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:34.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.007+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:34.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.007+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:34.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.007+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:34.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:34.165+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:34.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:34.165+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:34.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:34.165+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:34.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:34.165+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:34.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:34.063+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:34.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:34.063+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:34.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:34.063+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:34.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:34.063+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:34.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:34 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:34.063+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:34.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:34.429+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:34.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:34.429+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:34.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:34.429+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:34.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:34.429+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:34.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:34.429+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:35.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:35.162+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:35.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:35.162+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:35.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:35.162+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:35.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:35.162+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:35.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.965+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:35.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.965+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:35.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.965+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:35.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.965+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:35.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.965+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:35.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:34 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:34.965+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.450939+0000 front 2023-12-11T19:39:13.450966+0000 (oldest deadline 2023-12-11T19:39:34.551232+0000) 2023-12-11T19:39:35.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:35.024+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:35.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:35.024+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:35.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:35.024+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:35.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:35.024+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:35.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:35.024+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:35.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:35 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:35.024+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:35.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:35.395+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:35.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:35.395+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:35.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:35.395+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:35.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:35.395+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:35.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:35 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:35.395+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:36.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:36.125+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:36.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:36.125+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:36.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:36.125+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:36.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:36.125+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:36.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:35.963+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:36.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:35.963+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:36.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:35.963+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:36.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:35.963+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:36.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:35.963+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:36.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:35 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:35.963+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.450939+0000 front 2023-12-11T19:39:13.450966+0000 (oldest deadline 2023-12-11T19:39:34.551232+0000) 2023-12-11T19:39:36.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:36.038+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:36.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:36.038+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:36.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:36.038+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:36.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:36.038+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:36.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:36.038+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:36.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:36 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:36.038+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:36.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:36.353+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:36.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:36.353+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:36.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:36.353+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:36.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:36.353+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:36.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:36 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:36.353+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:37.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:37.114+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:37.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:37.114+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:37.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:37.114+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:37.269 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:37.114+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:37.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:36.955+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:37.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:36.955+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:37.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:36.955+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:37.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:36.955+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:37.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:36.955+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:37.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:36 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:36.955+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.450939+0000 front 2023-12-11T19:39:13.450966+0000 (oldest deadline 2023-12-11T19:39:34.551232+0000) 2023-12-11T19:39:37.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:37.080+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:37.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:37.080+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:37.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:37.080+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:37.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:37.080+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:37.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:37.080+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:37.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:37 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:37.080+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:37.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:37.306+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:37.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:37.306+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:37.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:37.306+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:37.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:37.306+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:37.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:37.306+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:37.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:37 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:37.306+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:38.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:37.959+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:30.942943+0000 front 2023-12-11T19:38:30.942783+0000 (oldest deadline 2023-12-11T19:38:54.442998+0000) 2023-12-11T19:39:38.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:37.959+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:40.244678+0000 front 2023-12-11T19:38:40.244840+0000 (oldest deadline 2023-12-11T19:39:03.144677+0000) 2023-12-11T19:39:38.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:37.959+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.647157+0000 front 2023-12-11T19:38:47.647136+0000 (oldest deadline 2023-12-11T19:39:10.547296+0000) 2023-12-11T19:39:38.269 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:37.959+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:56.448508+0000 front 2023-12-11T19:38:56.448524+0000 (oldest deadline 2023-12-11T19:39:19.348450+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:37.959+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.249904+0000 front 2023-12-11T19:39:02.249980+0000 (oldest deadline 2023-12-11T19:39:27.549924+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:37 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8[124924]: 2023-12-11T19:39:37.959+0000 7ffa1a0df700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.450939+0000 front 2023-12-11T19:39:13.450966+0000 (oldest deadline 2023-12-11T19:39:34.551232+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:38 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:38.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:38 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:38.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:38 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:38.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:38 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:38.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:38.270 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:38 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:38.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:38.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:38.055+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:38.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:38.055+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:38.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:38.055+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:38.313 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:38.055+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:38.313 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:38.055+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:38.314 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:38 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:38.055+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:38.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:38.287+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:38.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:38.287+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:38.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:38.287+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:38.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:38.287+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:38.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:38.287+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:38.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:38 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:38.287+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:39.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:39.047+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:39.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:39.047+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:39.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:39.047+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:39.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:39.047+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:39.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:39.047+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:39.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:39 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:39.047+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:39.367 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:39 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:39.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:39.367 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:39 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:39.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:39.367 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:39 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:39.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:39.368 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:39 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:39.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:39.368 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:39 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:39.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:39.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:39 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:39.338+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:39.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:39 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:39.338+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:39.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:39 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:39.338+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:39.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:39 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:39.338+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:39.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:39 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:39.338+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:39.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:39 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:39.338+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:40.019 INFO:journalctl@ceph.osd.8.smithi167.stdout:Dec 11 19:39:39 smithi167 bash[144324]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-8 2023-12-11T19:39:40.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:40.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:40.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:40.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:40.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:40.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:40.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:40.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:40.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:40.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:40.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:40 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:40.037+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:40.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:40 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:40.136+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:40.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:40 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:40.136+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:40.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:40 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:40.136+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:40.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:40 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:40.136+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:40.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:40 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:40.136+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:40.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:40.356+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:40.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:40.356+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:40.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:40.356+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:40.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:40.356+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:40.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:40.356+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:40.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:40.356+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:41.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:41.045+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:41.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:41.045+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:41.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:41.045+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:41.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:41.045+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:41.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:41.045+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:41.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:41.045+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:41.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:41 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:41.169+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:41.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:41 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:41.169+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:41.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:41 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:41.169+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:41.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:41 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:41.169+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:41.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:41 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:41.169+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:41.592 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.8.service' 2023-12-11T19:39:41.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:41 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:41.320+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:41.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:41 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:41.320+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:41.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:41 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:41.320+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:41.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:41 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:41.320+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:41.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:41 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:41.320+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:41.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:41 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:41.320+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:41.680 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:41.681 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-11T19:39:41.681 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-11T19:39:41.681 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.9 2023-12-11T19:39:42.053 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:41 smithi066 systemd[1]: Stopping Ceph osd.9 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:42.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.053+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.053+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.053+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.053+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.053+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.053+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.209+0000 7fdb99d2e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.209+0000 7fdb99d2e700 -1 osd.9 88 *** Got signal Terminated *** 2023-12-11T19:39:42.312 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:42 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:42.209+0000 7fdb99d2e700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:42.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:42 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:42.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:42.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:42 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:42.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:42.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:42 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:42.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:42.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:42 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:42.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:42.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:42 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:42.193+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:42.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:42 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:42.292+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:42.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:42 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:42.292+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:42.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:42 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:42.292+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:42.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:42 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:42.292+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:42.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:42 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:42.292+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:42.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:42 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:42.292+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:43.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:43.167+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:43.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:43.167+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:43.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:43.167+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:43.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:43.167+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:43.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:43 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:43.167+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:43.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:43 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:43.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:43.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:43 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:43.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:43.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:43 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:43.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:43.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:43 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:43.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:43.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:43 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:43.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:43.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:43 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:43.073+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:43.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:43 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:43.310+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:43.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:43 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:43.310+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:43.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:43 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:43.310+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:43.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:43 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:43.310+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:43.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:43 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:43.310+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:43.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:43 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:43.310+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:44.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:44 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:44.154+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:44.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:44 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:44.154+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:44.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:44 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:44.154+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:44.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:44 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:44.154+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:44.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:44 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:44.154+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:44.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:44 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:44.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:44.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:44 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:44.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:44.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:44 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:44.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:44.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:44 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:44.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:44.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:44 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:44.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:44.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:44 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:44.068+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:44.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:44 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:44.312+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:44.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:44 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:44.312+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:44.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:44 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:44.312+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:44.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:44 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:44.312+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:44.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:44 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:44.312+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:44.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:44 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:44.312+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:45.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:45 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:45.196+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:45.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:45 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:45.197+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:45.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:45 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:45.197+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:45.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:45 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:45.197+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:45.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:45 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:45.197+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:45.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:45 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:45.112+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:45.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:45 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:45.112+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:45.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:45 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:45.112+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:45.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:45 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:45.112+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:45.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:45 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:45.112+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:45.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:45 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:45.112+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:45.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:45.300+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:45.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:45.300+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:45.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:45.300+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:45.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:45.300+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:45.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:45.300+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:45.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:45.300+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:46.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:46 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:46.245+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:46.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:46 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:46.245+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:46.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:46 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:46.245+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:46.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:46 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:46.245+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:46.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:46 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:46.245+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:46.561 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:46.142+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:46.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:46.142+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:46.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:46.142+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:46.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:46.142+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:46.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:46.142+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:46.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:46 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:46.142+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:46.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:46 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:46.294+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:46.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:46 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:46.294+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:46.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:46 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:46.294+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:46.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:46 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:46.294+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:46.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:46 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:46.294+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:46.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:46 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:46.294+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:47.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:47 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:47.251+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:47.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:47 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:47.251+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:47.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:47 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:47.251+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:47.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:47 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:47.251+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:47.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:47 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:47.251+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:47.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:47 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:47.251+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:47.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:47.188+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.263057+0000 front 2023-12-11T19:38:31.263032+0000 (oldest deadline 2023-12-11T19:38:57.163311+0000) 2023-12-11T19:39:47.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:47.188+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:37.163951+0000 front 2023-12-11T19:38:37.164267+0000 (oldest deadline 2023-12-11T19:39:01.263995+0000) 2023-12-11T19:39:47.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:47.188+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.265955+0000 front 2023-12-11T19:38:48.265837+0000 (oldest deadline 2023-12-11T19:39:12.365611+0000) 2023-12-11T19:39:47.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:47.188+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:55.266752+0000 front 2023-12-11T19:38:55.266777+0000 (oldest deadline 2023-12-11T19:39:18.166901+0000) 2023-12-11T19:39:47.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:47.188+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:03.869529+0000 front 2023-12-11T19:39:03.869501+0000 (oldest deadline 2023-12-11T19:39:26.669872+0000) 2023-12-11T19:39:47.562 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9[137537]: 2023-12-11T19:39:47.188+0000 7fdb92321700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:10.071976+0000 front 2023-12-11T19:39:10.071939+0000 (oldest deadline 2023-12-11T19:39:34.771837+0000) 2023-12-11T19:39:47.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:47.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:47.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:47.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:47.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:47.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:47.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:47 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:47.269+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:48.311 INFO:journalctl@ceph.osd.9.smithi066.stdout:Dec 11 19:39:47 smithi066 bash[167196]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-9 2023-12-11T19:39:48.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:48 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:48.209+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:48.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:48 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:48.209+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:48.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:48 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:48.209+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:48.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:48 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:48.209+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:48.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:48 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:48.209+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:48.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:48 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:48.209+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.243+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.243+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.243+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.243+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:48.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:48 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:48.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:49.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:49.164+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:49.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:49.164+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:49.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:49.164+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:49.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:49.164+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:49.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:49.164+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:49.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:49.164+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:49.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:49 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:49.236+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:49.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.9.service' 2023-12-11T19:39:49.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:49.924 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-11T19:39:49.924 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-11T19:39:49.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.10 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 systemd[1]: Stopping Ceph osd.10 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:50.359 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.241+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:50.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:50 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:50.146+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:50.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:50 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:50.146+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:50.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:50 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:50.146+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:50.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:50 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:50.146+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:50.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:50 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:50.146+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:50.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:50 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:50.146+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:50.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.458+0000 7f4a53136700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:50.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.458+0000 7f4a53136700 -1 osd.10 88 *** Got signal Terminated *** 2023-12-11T19:39:50.859 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:50 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:50.458+0000 7f4a53136700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:51.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:51.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:51.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:51.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:51.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:51.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:51.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:51.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:51.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:51.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:51.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:51 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:51.106+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:51.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:51 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:51.232+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:52.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:52 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:52.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:52.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:52 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:52.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:52.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:52 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:52.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:52.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:52 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:52.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:52.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:52 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:52.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:52.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:52 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:52.126+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:52.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:52 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:52.244+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:53.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:53 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:53.080+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:53.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:53 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:53.080+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:53.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:53 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:53.080+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:53.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:53 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:53.080+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:53.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:53 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:53.080+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:53.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:53 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:53.080+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:53.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:53.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:53.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:53.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:53.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:53.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:53.610 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:53 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:53.201+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:54.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:54.085+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:54.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:54.085+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:54.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:54.085+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:54.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:54.085+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:54.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:54.085+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:54.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:54.085+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:54.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:54 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:54.202+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:55.522 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:55.522 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:55.523 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:55.523 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:55.523 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:55.523 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:55.523 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:55 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:55.064+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:27.209401+0000 front 2023-12-11T19:38:27.209589+0000 (oldest deadline 2023-12-11T19:38:57.210004+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:38.911254+0000 front 2023-12-11T19:38:38.911293+0000 (oldest deadline 2023-12-11T19:39:01.811387+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:47.613637+0000 front 2023-12-11T19:38:47.613402+0000 (oldest deadline 2023-12-11T19:39:12.313297+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-12-11T19:38:54.614467+0000 front 2023-12-11T19:38:54.614406+0000 (oldest deadline 2023-12-11T19:39:18.114402+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:05.115760+0000 front 2023-12-11T19:39:05.115728+0000 (oldest deadline 2023-12-11T19:39:26.116199+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:11.417618+0000 front 2023-12-11T19:39:11.417591+0000 (oldest deadline 2023-12-11T19:39:36.717266+0000) 2023-12-11T19:39:55.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:55 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10[133883]: 2023-12-11T19:39:55.159+0000 7f4a4b729700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.919215+0000 front 2023-12-11T19:39:21.919326+0000 (oldest deadline 2023-12-11T19:39:47.218965+0000) 2023-12-11T19:39:56.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:56.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:56.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:56.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:56.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:56.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:56.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:56 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:56.066+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:39:56.609 INFO:journalctl@ceph.osd.10.smithi070.stdout:Dec 11 19:39:56 smithi070 bash[144807]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-10 2023-12-11T19:39:57.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:57.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:57.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:57.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:57.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:57.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:57.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:57 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:57.104+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:39:58.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.10.service' 2023-12-11T19:39:58.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:39:58.266 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-11T19:39:58.266 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-11T19:39:58.266 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.11 2023-12-11T19:39:58.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.145+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:39:58.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 systemd[1]: Stopping Ceph osd.11 for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:39:59.175 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.780+0000 7fbd7e4b4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:39:59.175 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.780+0000 7fbd7e4b4700 -1 osd.11 88 *** Got signal Terminated *** 2023-12-11T19:39:59.175 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:58.780+0000 7fbd7e4b4700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T19:39:59.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:39:59.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:39:59.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:39:59.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:39:59.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:39:59.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:39:59.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:39:59 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:39:59.175+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:40:00.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:40:00.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:40:00.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:40:00.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:40:00.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:40:00.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:40:00.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:00 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:00.218+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:40:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:40:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:40:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:40:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:40:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:40:01.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:40:01.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:01 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:01.268+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:40:02.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:40:02.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:40:02.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:40:02.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:40:02.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:40:02.519 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:40:02.520 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:02 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:02.229+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-12-11T19:38:31.826466+0000 front 2023-12-11T19:38:31.826409+0000 (oldest deadline 2023-12-11T19:38:55.925862+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6804 osd.1 since back 2023-12-11T19:38:35.926643+0000 front 2023-12-11T19:38:35.926413+0000 (oldest deadline 2023-12-11T19:39:01.826491+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-12-11T19:38:48.728881+0000 front 2023-12-11T19:38:48.729141+0000 (oldest deadline 2023-12-11T19:39:12.229051+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6812 osd.4 since back 2023-12-11T19:39:02.131625+0000 front 2023-12-11T19:39:02.131723+0000 (oldest deadline 2023-12-11T19:39:31.532393+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-12-11T19:39:13.233442+0000 front 2023-12-11T19:39:13.233619+0000 (oldest deadline 2023-12-11T19:39:37.333582+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-12-11T19:39:21.434819+0000 front 2023-12-11T19:39:21.434692+0000 (oldest deadline 2023-12-11T19:39:46.734699+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.70:6820 osd.7 since back 2023-12-11T19:39:30.235779+0000 front 2023-12-11T19:39:30.235870+0000 (oldest deadline 2023-12-11T19:39:54.335758+0000) 2023-12-11T19:40:03.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:03 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11[133608]: 2023-12-11T19:40:03.278+0000 7fbd782aa700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2023-12-11T19:39:37.236779+0000 front 2023-12-11T19:39:37.236821+0000 (oldest deadline 2023-12-11T19:40:02.536933+0000) 2023-12-11T19:40:04.769 INFO:journalctl@ceph.osd.11.smithi167.stdout:Dec 11 19:40:04 smithi167 bash[144589]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-osd-11 2023-12-11T19:40:06.321 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@osd.11.service' 2023-12-11T19:40:06.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:06.410 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-11T19:40:06.410 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-11T19:40:06.411 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.a 2023-12-11T19:40:06.812 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:40:06 smithi066 systemd[1]: Stopping Ceph mds.a for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:07.311 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:40:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-a[145263]: 2023-12-11T19:40:06.932+0000 7fc85e490700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:07.311 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:40:06 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-a[145263]: 2023-12-11T19:40:06.932+0000 7fc85e490700 -1 mds.a *** got signal Terminated *** 2023-12-11T19:40:11.811 INFO:journalctl@ceph.mds.a.smithi066.stdout:Dec 11 19:40:11 smithi066 bash[167469]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-a 2023-12-11T19:40:11.854 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.a.service' 2023-12-11T19:40:11.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:11.942 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-11T19:40:11.942 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-11T19:40:11.942 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.d 2023-12-11T19:40:12.311 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:40:12 smithi066 systemd[1]: Stopping Ceph mds.d for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:12.812 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:40:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-d[144176]: 2023-12-11T19:40:12.464+0000 7f29458ce700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:12.812 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:40:12 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-d[144176]: 2023-12-11T19:40:12.464+0000 7f29458ce700 -1 mds.d *** got signal Terminated *** 2023-12-11T19:40:16.543 INFO:journalctl@ceph.mds.d.smithi066.stdout:Dec 11 19:40:16 smithi066 bash[167612]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-d 2023-12-11T19:40:16.864 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.d.service' 2023-12-11T19:40:16.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:16.951 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-11T19:40:16.951 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-11T19:40:16.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.g 2023-12-11T19:40:17.311 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:40:17 smithi066 systemd[1]: Stopping Ceph mds.g for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:17.811 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:40:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-g[144870]: 2023-12-11T19:40:17.463+0000 7fe1c5859700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:17.812 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:40:17 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-g[144870]: 2023-12-11T19:40:17.463+0000 7fe1c5859700 -1 mds.g *** got signal Terminated *** 2023-12-11T19:40:20.062 INFO:journalctl@ceph.mds.g.smithi066.stdout:Dec 11 19:40:19 smithi066 bash[167754]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-g 2023-12-11T19:40:20.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.g.service' 2023-12-11T19:40:20.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:20.389 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-11T19:40:20.390 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-11T19:40:20.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.j 2023-12-11T19:40:20.562 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:40:20 smithi066 systemd[1]: Stopping Ceph mds.j for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:21.311 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:40:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-j[143791]: 2023-12-11T19:40:20.896+0000 7f7f9f410700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:21.312 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:40:20 smithi066 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-j[143791]: 2023-12-11T19:40:20.896+0000 7f7f9f410700 -1 mds.j *** got signal Terminated *** 2023-12-11T19:40:25.478 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:40:25 smithi066 bash[167900]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-j 2023-12-11T19:40:25.706 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.j.service' 2023-12-11T19:40:25.768 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:40:25 smithi066 systemd[1]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.j.service: Succeeded. 2023-12-11T19:40:25.768 INFO:journalctl@ceph.mds.j.smithi066.stdout:Dec 11 19:40:25 smithi066 systemd[1]: Stopped Ceph mds.j for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:40:25.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:25.792 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-11T19:40:25.792 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-11T19:40:25.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.b 2023-12-11T19:40:26.109 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:40:25 smithi070 systemd[1]: Stopping Ceph mds.b for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:26.609 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:40:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b[140194]: 2023-12-11T19:40:26.325+0000 7f664a586700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:26.609 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:40:26 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b[140194]: 2023-12-11T19:40:26.325+0000 7f664a586700 -1 mds.b *** got signal Terminated *** 2023-12-11T19:40:34.281 INFO:journalctl@ceph.mds.b.smithi070.stdout:Dec 11 19:40:33 smithi070 bash[145077]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-b 2023-12-11T19:40:34.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.b.service' 2023-12-11T19:40:34.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:34.392 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-11T19:40:34.392 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-11T19:40:34.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.e 2023-12-11T19:40:34.609 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:40:34 smithi070 systemd[1]: Stopping Ceph mds.e for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:35.359 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:40:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-e[139499]: 2023-12-11T19:40:34.906+0000 7f0ad632d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:35.359 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:40:34 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-e[139499]: 2023-12-11T19:40:34.906+0000 7f0ad632d700 -1 mds.e *** got signal Terminated *** 2023-12-11T19:40:39.109 INFO:journalctl@ceph.mds.e.smithi070.stdout:Dec 11 19:40:38 smithi070 bash[145216]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-e 2023-12-11T19:40:39.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.e.service' 2023-12-11T19:40:39.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:39.486 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-11T19:40:39.486 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-11T19:40:39.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.h 2023-12-11T19:40:39.859 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:40:39 smithi070 systemd[1]: Stopping Ceph mds.h for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:40.359 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:40:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-h[139842]: 2023-12-11T19:40:40.006+0000 7f2afddf5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:40.359 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:40:40 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-h[139842]: 2023-12-11T19:40:40.006+0000 7f2afddf5700 -1 mds.h *** got signal Terminated *** 2023-12-11T19:40:44.858 INFO:journalctl@ceph.mds.h.smithi070.stdout:Dec 11 19:40:44 smithi070 bash[145368]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-h 2023-12-11T19:40:44.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.h.service' 2023-12-11T19:40:45.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:45.050 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-11T19:40:45.050 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-11T19:40:45.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.k 2023-12-11T19:40:45.358 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:40:45 smithi070 systemd[1]: Stopping Ceph mds.k for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:45.858 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:40:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-k[139159]: 2023-12-11T19:40:45.545+0000 7fadf5416700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:45.859 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:40:45 smithi070 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-k[139159]: 2023-12-11T19:40:45.545+0000 7fadf5416700 -1 mds.k *** got signal Terminated *** 2023-12-11T19:40:48.109 INFO:journalctl@ceph.mds.k.smithi070.stdout:Dec 11 19:40:48 smithi070 bash[145514]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-k 2023-12-11T19:40:48.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.k.service' 2023-12-11T19:40:48.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:48.511 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-11T19:40:48.511 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-11T19:40:48.511 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.c 2023-12-11T19:40:49.019 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:40:48 smithi167 systemd[1]: Stopping Ceph mds.c for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:49.519 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:40:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-c[139678]: 2023-12-11T19:40:49.025+0000 7fe5cd508700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:49.519 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:40:49 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-c[139678]: 2023-12-11T19:40:49.025+0000 7fe5cd508700 -1 mds.c *** got signal Terminated *** 2023-12-11T19:40:53.747 INFO:journalctl@ceph.mds.c.smithi167.stdout:Dec 11 19:40:53 smithi167 bash[144873]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-c 2023-12-11T19:40:53.776 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.c.service' 2023-12-11T19:40:53.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:53.863 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-11T19:40:53.863 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-11T19:40:53.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.f 2023-12-11T19:40:54.019 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:40:53 smithi167 systemd[1]: Stopping Ceph mds.f for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:54.769 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:40:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-f[139336]: 2023-12-11T19:40:54.347+0000 7fe66fede700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:54.769 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:40:54 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-f[139336]: 2023-12-11T19:40:54.347+0000 7fe66fede700 -1 mds.f *** got signal Terminated *** 2023-12-11T19:40:58.269 INFO:journalctl@ceph.mds.f.smithi167.stdout:Dec 11 19:40:57 smithi167 bash[145018]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-f 2023-12-11T19:40:58.349 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.f.service' 2023-12-11T19:40:58.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:40:58.431 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-11T19:40:58.432 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-11T19:40:58.432 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.i 2023-12-11T19:40:58.769 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:40:58 smithi167 systemd[1]: Stopping Ceph mds.i for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:40:59.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:40:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-i[140017]: 2023-12-11T19:40:58.930+0000 7f1539d3d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:40:59.269 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:40:58 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-i[140017]: 2023-12-11T19:40:58.930+0000 7f1539d3d700 -1 mds.i *** got signal Terminated *** 2023-12-11T19:41:05.496 INFO:journalctl@ceph.mds.i.smithi167.stdout:Dec 11 19:41:05 smithi167 bash[145164]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-i 2023-12-11T19:41:05.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.i.service' 2023-12-11T19:41:05.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:41:05.612 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-11T19:41:05.612 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-11T19:41:05.612 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.l 2023-12-11T19:41:05.769 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:41:05 smithi167 systemd[1]: Stopping Ceph mds.l for 976cf7f8-985a-11ee-95a3-87774f69a715... 2023-12-11T19:41:06.519 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:41:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-l[138998]: 2023-12-11T19:41:06.129+0000 7f789f4d2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T19:41:06.519 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:41:06 smithi167 ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-l[138998]: 2023-12-11T19:41:06.129+0000 7f789f4d2700 -1 mds.l *** got signal Terminated *** 2023-12-11T19:41:10.860 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:41:10 smithi167 bash[145314]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715-mds-l 2023-12-11T19:41:11.075 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.l.service' 2023-12-11T19:41:11.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:41:11 smithi167 systemd[1]: ceph-976cf7f8-985a-11ee-95a3-87774f69a715@mds.l.service: Succeeded. 2023-12-11T19:41:11.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Dec 11 19:41:11 smithi167 systemd[1]: Stopped Ceph mds.l for 976cf7f8-985a-11ee-95a3-87774f69a715. 2023-12-11T19:41:11.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T19:41:11.159 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-11T19:41:11.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-11T19:41:11.620 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:41:15.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-11T19:41:15.515 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:41:18.723 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-11T19:41:19.161 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:41:22.486 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:41:22.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:41:22.648 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T19:41:22.719 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-11T19:41:22.721 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi066/crash 2023-12-11T19:41:22.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash -- . 2023-12-11T19:41:22.807 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-11T19:41:22.807 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2023-12-11T19:41:22.811 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi070/crash 2023-12-11T19:41:22.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash -- . 2023-12-11T19:41:22.902 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-11T19:41:22.902 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2023-12-11T19:41:22.905 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi167/crash 2023-12-11T19:41:22.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash -- . 2023-12-11T19:41:22.990 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-11T19:41:22.990 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2023-12-11T19:41:22.993 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-11T19:41:22.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-12-11T19:41:23.081 INFO:teuthology.orchestra.run.smithi066.stderr:grep: /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph.log: No such file or directory 2023-12-11T19:41:23.084 INFO:tasks.cephadm:Compressing logs... 2023-12-11T19:41:23.084 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-11T19:41:23.089 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-11T19:41:23.094 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-11T19:41:23.155 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-11T19:41:23.155 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-11T19:41:23.157 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.0.log 2023-12-11T19:41:23.158 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.g.log 2023-12-11T19:41:23.159 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.a.log 2023-12-11T19:41:23.160 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.6.log 2023-12-11T19:41:23.161 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-11T19:41:23.161 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-11T19:41:23.162 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.3.log 2023-12-11T19:41:23.162 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.10.log 2023-12-11T19:41:23.163 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.h.log 2023-12-11T19:41:23.163 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.d.log 2023-12-11T19:41:23.163 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-11T19:41:23.164 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.10.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-11T19:41:23.165 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.e.log 2023-12-11T19:41:23.166 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.x.log 2023-12-11T19:41:23.166 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.4.log 2023-12-11T19:41:23.170 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.1.log 2023-12-11T19:41:23.171 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.k.log 2023-12-11T19:41:23.175 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.9.log 2023-12-11T19:41:23.175 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.x.log: /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.d.log.gz 2023-12-11T19:41:23.175 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-11T19:41:23.175 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.j.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.5.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.2.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.11.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.z.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.8.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.f.log 2023-12-11T19:41:23.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.c.log 2023-12-11T19:41:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-11T19:41:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.7.log 2023-12-11T19:41:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log: /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.b.log 2023-12-11T19:41:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.b.log 2023-12-11T19:41:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log: /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.a.log 2023-12-11T19:41:23.181 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.f.log: 92.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.z.log.gz 2023-12-11T19:41:23.182 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.i.log 2023-12-11T19:41:23.182 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.c.log 2023-12-11T19:41:23.185 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-11T19:41:23.186 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.b.log: /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.y.log 2023-12-11T19:41:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2023-12-11T19:41:23.187 INFO:teuthology.orchestra.run.smithi070.stderr: 91.1% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-11T19:41:23.188 INFO:teuthology.orchestra.run.smithi066.stderr: 94.7% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.j.log.gz 2023-12-11T19:41:23.195 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.c.log: /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.l.log 2023-12-11T19:41:23.196 INFO:teuthology.orchestra.run.smithi066.stderr: 94.4%/var/log/ceph/ceph-client.0.log: -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.g.log.gz 2023-12-11T19:41:23.196 INFO:teuthology.orchestra.run.smithi066.stderr: 94.8% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.a.log.gz 2023-12-11T19:41:23.196 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-11T19:41:23.197 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.y.log: 93.1% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.e.log.gz 2023-12-11T19:41:23.197 INFO:teuthology.orchestra.run.smithi070.stderr: 92.3% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.y.log.gz 2023-12-11T19:41:23.210 INFO:teuthology.orchestra.run.smithi070.stderr: 93.7% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.h.log.gz 2023-12-11T19:41:23.210 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.l.log: 93.1% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.f.log.gz 2023-12-11T19:41:23.213 INFO:teuthology.orchestra.run.smithi167.stderr: 91.1% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-11T19:41:23.213 INFO:teuthology.orchestra.run.smithi167.stderr: 93.1% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.i.log.gz 2023-12-11T19:41:23.219 INFO:teuthology.orchestra.run.smithi070.stderr: 93.0% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.k.log.gz 2023-12-11T19:41:23.226 INFO:teuthology.orchestra.run.smithi167.stderr: 94.9% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.c.log.gz 2023-12-11T19:41:23.232 INFO:teuthology.orchestra.run.smithi167.stderr: 94.8% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.l.log.gz 2023-12-11T19:41:23.383 INFO:teuthology.orchestra.run.smithi066.stderr: 89.6% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mgr.x.log.gz 2023-12-11T19:41:23.404 INFO:teuthology.orchestra.run.smithi167.stderr: 92.3% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.c.log.gz 2023-12-11T19:41:23.422 INFO:teuthology.orchestra.run.smithi070.stderr: 92.0% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.b.log.gz 2023-12-11T19:41:23.453 INFO:teuthology.orchestra.run.smithi070.stderr: 92.7% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mds.b.log.gz 2023-12-11T19:41:24.112 INFO:teuthology.orchestra.run.smithi066.stderr: 90.6% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-mon.a.log.gz 2023-12-11T19:41:25.971 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.9.log.gz 2023-12-11T19:41:26.352 INFO:teuthology.orchestra.run.smithi167.stderr: 93.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.8.log.gz 2023-12-11T19:41:26.466 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.7.log.gz 2023-12-11T19:41:26.761 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.11.log.gz 2023-12-11T19:41:26.767 INFO:teuthology.orchestra.run.smithi167.stderr: 93.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.2.log.gz 2023-12-11T19:41:26.872 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.10.log.gz 2023-12-11T19:41:26.961 INFO:teuthology.orchestra.run.smithi070.stderr: 93.5% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.4.log.gz 2023-12-11T19:41:27.011 INFO:teuthology.orchestra.run.smithi167.stderr: 93.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.5.log.gz 2023-12-11T19:41:27.014 INFO:teuthology.orchestra.run.smithi167.stderr: 2023-12-11T19:41:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m3.874s 2023-12-11T19:41:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m13.443s 2023-12-11T19:41:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m1.194s 2023-12-11T19:41:27.289 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.3.log.gz 2023-12-11T19:41:27.332 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.1.log.gz 2023-12-11T19:41:27.336 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-12-11T19:41:27.336 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m4.202s 2023-12-11T19:41:27.336 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m14.374s 2023-12-11T19:41:27.336 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m1.276s 2023-12-11T19:41:27.455 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.6.log.gz 2023-12-11T19:41:27.462 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/976cf7f8-985a-11ee-95a3-87774f69a715/ceph-osd.0.log.gz 2023-12-11T19:41:32.701 INFO:teuthology.orchestra.run.smithi066.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2023-12-11T19:41:32.702 INFO:teuthology.orchestra.run.smithi066.stderr: 2023-12-11T19:41:32.702 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m9.576s 2023-12-11T19:41:32.702 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m24.321s 2023-12-11T19:41:32.703 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m2.017s 2023-12-11T19:41:32.704 INFO:tasks.cephadm:Archiving logs... 2023-12-11T19:41:32.704 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi066/log 2023-12-11T19:41:32.705 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-11T19:41:35.861 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi070/log 2023-12-11T19:41:35.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-11T19:41:38.120 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi167/log 2023-12-11T19:41:38.120 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-11T19:41:40.255 INFO:tasks.cephadm:Removing cluster... 2023-12-11T19:41:40.255 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --force 2023-12-11T19:41:40.712 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:41:41.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --force 2023-12-11T19:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:41:42.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 976cf7f8-985a-11ee-95a3-87774f69a715 --force 2023-12-11T19:41:43.166 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 976cf7f8-985a-11ee-95a3-87774f69a715 2023-12-11T19:41:43.857 INFO:tasks.cephadm:Removing cephadm ... 2023-12-11T19:41:43.857 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-11T19:41:43.909 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-11T19:41:43.958 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-11T19:41:44.003 INFO:tasks.cephadm:Teardown complete 2023-12-11T19:41:44.005 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-11T19:41:44.017 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-12-11T19:41:44.018 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-11T19:41:44.018 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-11T19:41:44.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-11T19:41:44.029 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-11T19:41:44.186 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-11T19:41:44.196 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-11T19:41:44.197 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T19:41:44.201 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T19:41:44.206 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T19:41:44.241 INFO:teuthology.orchestra.run.smithi066.stderr:bash: ntpq: command not found 2023-12-11T19:41:44.243 INFO:teuthology.orchestra.run.smithi070.stderr:bash: ntpq: command not found 2023-12-11T19:41:44.247 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-12-11T19:41:44.250 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T19:41:44.250 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2023-12-11T19:41:44.250 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 +2162us[+2162us] +/- 88ms 2023-12-11T19:41:44.250 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -1788us[-1793us] +/- 75ms 2023-12-11T19:41:44.250 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -1529ns[-1529ns] +/- 85ms 2023-12-11T19:41:44.250 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:41:44.251 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T19:41:44.251 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-12-11T19:41:44.251 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +2147us[+2146us] +/- 88ms 2023-12-11T19:41:44.251 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -1788us[-1789us] +/- 75ms 2023-12-11T19:41:44.251 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -22us[ -24us] +/- 85ms 2023-12-11T19:41:44.252 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:41:44.255 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T19:41:44.255 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-12-11T19:41:44.255 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 +2114us[+2114us] +/- 88ms 2023-12-11T19:41:44.255 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -1813us[-1778us] +/- 75ms 2023-12-11T19:41:44.255 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -36us[ -36us] +/- 85ms 2023-12-11T19:41:44.255 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T19:41:44.256 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-11T19:41:44.267 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-11T19:41:44.268 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-11T19:41:44.286 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-11T19:41:44.474 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-11T19:41:44.638 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-11T19:41:44.806 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T19:41:44.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:41:44.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T19:41:44.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:41:44.963 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T19:41:45.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:41:45.038 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-11T19:41:45.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-11T19:41:45.067 INFO:teuthology.task.internal:Duration was 2134.003214 seconds 2023-12-11T19:41:45.068 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-11T19:41:45.084 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-11T19:41:45.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-11T19:41:45.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-11T19:41:45.094 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-11T19:41:45.206 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T19:41:45.208 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T19:41:45.212 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T19:41:45.600 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-11T19:41:45.600 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2023-12-11T19:41:45.600 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-11T19:41:45.661 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2023-12-11T19:41:45.662 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-11T19:41:45.722 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2023-12-11T19:41:45.722 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-11T19:41:45.781 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-11T19:41:45.781 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-11T19:41:45.786 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-11T19:41:45.791 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-11T19:41:45.883 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-11T19:41:45.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-11T19:41:45.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-11T19:41:45.893 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-11T19:41:46.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-11T19:41:46.060 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-11T19:41:46.060 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-11T19:41:46.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-11T19:41:46.201 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-11T19:41:46.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-11T19:41:46.285 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-11T19:41:46.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-11T19:41:46.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-11T19:41:46.355 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2023-12-11T19:41:46.358 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2023-12-11T19:41:46.362 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2023-12-11T19:41:46.408 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-11T19:41:46.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:41:46.450 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-11T19:41:46.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:41:46.493 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-11T19:41:46.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T19:41:46.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-11T19:41:46.547 INFO:teuthology.task.internal:Transferring archived files... 2023-12-11T19:41:46.548 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi066 2023-12-11T19:41:46.548 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-11T19:41:46.637 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi070 2023-12-11T19:41:46.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-11T19:41:46.717 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203/remote/smithi167 2023-12-11T19:41:46.717 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-11T19:41:46.799 INFO:teuthology.task.internal:Removing archive directory... 2023-12-11T19:41:46.799 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-11T19:41:46.804 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-11T19:41:46.808 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-11T19:41:46.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-11T19:41:46.864 INFO:teuthology.task.internal:Not uploading archives. 2023-12-11T19:41:46.864 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-11T19:41:46.878 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-11T19:41:46.878 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-11T19:41:46.889 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-11T19:41:46.893 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-11T19:41:46.929 INFO:teuthology.orchestra.run.smithi066.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 11 19:41 /home/ubuntu/cephtest 2023-12-11T19:41:46.932 INFO:teuthology.orchestra.run.smithi070.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 11 19:41 /home/ubuntu/cephtest 2023-12-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi167.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 11 19:41 /home/ubuntu/cephtest 2023-12-11T19:41:46.940 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-11T19:41:46.954 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-11T19:41:47.000 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-12-11T19:41:47.000 INFO:teuthology.nuke:Checking targets against current locks 2023-12-11T19:41:47.027 DEBUG:teuthology.nuke:shortname: smithi066 2023-12-11T19:41:47.027 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T19:41:47.058 DEBUG:teuthology.nuke:shortname: smithi070 2023-12-11T19:41:47.058 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T19:41:47.063 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 18:54:46.448960', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T19:41:47.081 DEBUG:teuthology.nuke:shortname: smithi167 2023-12-11T19:41:47.082 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T19:41:47.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 18:54:46.450660', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T19:41:47.112 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 18:54:46.452004', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T19:41:47.120 INFO:teuthology.orchestra.console.smithi066:Power off 2023-12-11T19:41:47.120 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-11T19:41:47.147 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2023-12-11T19:41:47.148 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:47.165 INFO:teuthology.orchestra.console.smithi070:Power off 2023-12-11T19:41:47.166 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-11T19:41:47.175 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2023-12-11T19:41:47.189 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2023-12-11T19:41:47.189 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:47.205 INFO:teuthology.orchestra.console.smithi167:Power off 2023-12-11T19:41:47.205 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-11T19:41:47.216 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-12-11T19:41:47.230 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-12-11T19:41:47.230 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:47.257 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-11T19:41:51.176 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:51.217 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:51.258 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:51.288 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2023-12-11T19:41:51.332 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-12-11T19:41:51.368 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-11T19:41:55.291 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:55.333 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:55.370 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:55.406 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2023-12-11T19:41:55.443 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-12-11T19:41:55.481 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-11T19:41:55.544 INFO:teuthology.orchestra.console.smithi070:Power off completed 2023-12-11T19:41:59.410 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:59.482 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T19:41:59.521 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2023-12-11T19:41:59.594 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-12-11T19:41:59.622 INFO:teuthology.orchestra.console.smithi066:Power off completed 2023-12-11T19:41:59.695 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-12-11T19:41:59.800 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} duration: 2134.0032138824463 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default mon.a-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c mon.b-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c mon.c-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c owner: scheduled_rishabh@teuthology status: fail success: false 2023-12-11T19:41:59.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-11T19:41:59.879 INFO:teuthology.run:FAIL